[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:06:46 dist-test-slave-2d19 krb5kdc[4030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:06:46 dist-test-slave-2d19 krb5kdc[4030](info): set up 2 sockets
Aug 11 07:06:46 dist-test-slave-2d19 krb5kdc[4030](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 07:06:47.984599 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.416s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:06:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:06:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:06:48.115559 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44979
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:44979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:48.203634 4051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:48.203846 4051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:48.203868 4051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:48.205077 4051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:06:48.205120 4051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:48.205135 4051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:48.205171 4051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:06:48.205204 4051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:06:48.206694 4051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:44979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:48.206954 4051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:48.207195 4051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:48.214362 4063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.214457 4062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.214362 4066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:48.215526 4051 server_base.cc:1034] running on GCE node
I20260811 07:06:48.215883 4051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:48.216239 4051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:48.217468 4051 hybrid_clock.cc:648] HybridClock initialized: now 1786432008217474 us; error 67 us; skew 500 ppm
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:48.220170 4051 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:06:48.220678 4051 webserver.cc:460] Webserver started at http://127.3.236.190:41381/ using document root <none> and password file <none>
I20260811 07:06:48.220916 4051 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:48.220985 4051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:48.221174 4051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:48.222353 4051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "df267ce30c9e450aab390f0bf7e21c62"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "30d657f78b6c515a116b6ddf447a1d60"
server_key_iv: "6d1fd0fc32a18ce1bd00368fd1ff2fea"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.222752 4051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "df267ce30c9e450aab390f0bf7e21c62"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "30d657f78b6c515a116b6ddf447a1d60"
server_key_iv: "6d1fd0fc32a18ce1bd00368fd1ff2fea"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.224195 4051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:06:48.224997 4075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.225243 4051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:48.225363 4051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "df267ce30c9e450aab390f0bf7e21c62"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "30d657f78b6c515a116b6ddf447a1d60"
server_key_iv: "6d1fd0fc32a18ce1bd00368fd1ff2fea"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.225442 4051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:48.238698 4051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:48.240994 4051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:48.241199 4051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:48.245940 4051 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:44979
I20260811 07:06:48.245977 4159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:44979 every 8 connection(s)
I20260811 07:06:48.246423 4051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:06:48.247548 4160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:48.250311 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62: Bootstrap starting.
I20260811 07:06:48.250991 4160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:48.251258 4160 log.cc:824] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:48.252023 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62: No bootstrap required, opened a new log
I20260811 07:06:48.252692 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4051
I20260811 07:06:48.252804 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:06:48.253592 4018 external_mini_cluster.cc:1428] Setting key 1afc7ddda1467b703b4147f56e50374a
I20260811 07:06:48.254673 4160 raft_consensus.cc:348] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } }
I20260811 07:06:48.254896 4160 raft_consensus.cc:374] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:48.255012 4160 raft_consensus.cc:729] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df267ce30c9e450aab390f0bf7e21c62, State: Initialized, Role: FOLLOWER
I20260811 07:06:48.255262 4160 consensus_queue.cc:260] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } }
I20260811 07:06:48.255373 4160 raft_consensus.cc:388] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:06:48.255411 4160 raft_consensus.cc:482] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:06:48.255503 4160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:48.256698 4160 raft_consensus.cc:504] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } }
I20260811 07:06:48.256830 4160 leader_election.cc:302] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df267ce30c9e450aab390f0bf7e21c62; no voters:
I20260811 07:06:48.257007 4160 leader_election.cc:288] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:06:48.257045 4170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:48.257229 4170 raft_consensus.cc:686] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [term 1 LEADER]: Becoming Leader. State: Replica: df267ce30c9e450aab390f0bf7e21c62, State: Running, Role: LEADER
I20260811 07:06:48.257352 4160 sys_catalog.cc:564] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:06:48.257395 4170 consensus_queue.cc:237] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } }
I20260811 07:06:48.257923 4171 sys_catalog.cc:455] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df267ce30c9e450aab390f0bf7e21c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } } }
I20260811 07:06:48.257941 4173 sys_catalog.cc:455] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df267ce30c9e450aab390f0bf7e21c62. Latest consensus state: current_term: 1 leader_uuid: "df267ce30c9e450aab390f0bf7e21c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df267ce30c9e450aab390f0bf7e21c62" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44979 } } }
I20260811 07:06:48.258122 4171 sys_catalog.cc:458] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:48.258195 4173 sys_catalog.cc:458] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:48.258517 4182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:06:48.259279 4182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:06:48.261082 4182 catalog_manager.cc:1347] Generated new cluster ID: afb6596b299945ee9be0a1ef12561d94
I20260811 07:06:48.261137 4182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:48.274015 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.254957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.3.236.190:44979)
0811 07:06:48.255166 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.255175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.255400 (+ 225us) server_negotiation.cc:407] Connection header received
0811 07:06:48.258073 (+ 2673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.258082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.258227 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.258362 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.261188 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.262784 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.264405 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.265052 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.265132 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.269729 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.269734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.269734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.269748 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.271828 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.272352 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.272356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.272356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.272478 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.272783 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.272784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.272785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.273169 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.273367 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.273631 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.273783 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260811 07:06:48.287762 4182 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:06:48.288616 4182 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:06:48.304098 4182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df267ce30c9e450aab390f0bf7e21c62: Generated new TSK 0
I20260811 07:06:48.304371 4182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:06:48.336084 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:48.423341 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:48.423557 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:48.423583 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:48.424769 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:48.424813 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:48.424877 4197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:06:48.426388 4197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:48.426704 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:48.426968 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:48.432526 4211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.432533 4214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.432533 4210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:48.432745 4197 server_base.cc:1034] running on GCE node
I20260811 07:06:48.433010 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:48.433262 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:48.434463 4197 hybrid_clock.cc:648] HybridClock initialized: now 1786432008434508 us; error 89 us; skew 500 ppm
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:48.437176 4197 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:06:48.437634 4197 webserver.cc:460] Webserver started at http://127.3.236.129:45633/ using document root <none> and password file <none>
I20260811 07:06:48.437880 4197 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:48.437954 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:48.438115 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:48.439254 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "23d84bb89efc42159c04cef014e1a2f2"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "50cfe6f6555781315b9142b96414ce0d"
server_key_iv: "a9a75768ef3de55fec0b932a85dee6a7"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.439642 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "23d84bb89efc42159c04cef014e1a2f2"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "50cfe6f6555781315b9142b96414ce0d"
server_key_iv: "a9a75768ef3de55fec0b932a85dee6a7"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.441154 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:06:48.441910 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.442098 4197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:48.442205 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "23d84bb89efc42159c04cef014e1a2f2"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "50cfe6f6555781315b9142b96414ce0d"
server_key_iv: "a9a75768ef3de55fec0b932a85dee6a7"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.442298 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:48.450420 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:48.452245 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:48.452461 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:48.452696 4197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:48.453052 4197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:48.453174 4197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.453217 4197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:48.453246 4197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.460170 4197 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:33119
I20260811 07:06:48.460254 4418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:33119 every 8 connection(s)
I20260811 07:06:48.460608 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:06:48.461592 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4197
I20260811 07:06:48.461696 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:06:48.461869 4018 external_mini_cluster.cc:1428] Setting key 7ae5ccdc7f7dab1b71bb68934e3ee427
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:06:48.471909 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.461513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:52475 (local address 127.3.236.190:44979)
0811 07:06:48.461602 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.461605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.461885 (+ 280us) server_negotiation.cc:407] Connection header received
0811 07:06:48.462641 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.462642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.462655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.462713 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.464325 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.465001 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.466164 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.466374 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.466466 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.468944 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.468946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.468946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.468954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.470443 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.470877 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.470879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.470879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.470942 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.471208 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.471209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.471210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.471336 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.471412 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.471708 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.471801 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:06:48.472115 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.461479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44979 (local address 127.3.236.129:52475)
0811 07:06:48.461728 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:48.461778 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:48.462455 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:48.462820 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:48.462825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:48.463913 (+ 1088us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:48.464215 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.464222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.465198 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.465199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.466060 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.466062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.466485 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.466486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.466627 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.467202 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:48.467210 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:48.468843 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:48.470546 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.470554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.470557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.470785 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.471024 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.471025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.471025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.471139 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.471415 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:48.471423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:48.471571 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 07:06:48.471684 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:06:48.471852 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":132,"threads_started":1}
I20260811 07:06:48.472708 4420 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44979
I20260811 07:06:48.472852 4420 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:48.473068 4420 heartbeater.cc:511] Master 127.3.236.190:44979 requested a full tablet report, sending...
I20260811 07:06:48.473832 4098 ts_manager.cc:194] Registered new tserver with Master: 23d84bb89efc42159c04cef014e1a2f2 (127.3.236.129:33119)
I20260811 07:06:48.474619 4098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:52475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:06:48.491266 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:48.581344 4428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:48.581602 4428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:48.581646 4428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:48.582827 4428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:48.582903 4428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:48.582952 4428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:06:48.584553 4428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:48.584818 4428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:48.585058 4428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:48.590415 4440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.590435 4437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.590425 4436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:48.590442 4428 server_base.cc:1034] running on GCE node
I20260811 07:06:48.590883 4428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:48.591176 4428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:48.592332 4428 hybrid_clock.cc:648] HybridClock initialized: now 1786432008592311 us; error 39 us; skew 500 ppm
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:48.595000 4428 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:06:48.595432 4428 webserver.cc:460] Webserver started at http://127.3.236.130:43071/ using document root <none> and password file <none>
I20260811 07:06:48.595667 4428 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:48.595736 4428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:48.595888 4428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:48.597035 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "967e1aecedbd4d51892507fe9d974c04"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "10fea70e8331f2b7a18776b37e02d0f5"
server_key_iv: "f8fba656e858a644e3913ff448b5cb4d"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.597446 4428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "967e1aecedbd4d51892507fe9d974c04"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "10fea70e8331f2b7a18776b37e02d0f5"
server_key_iv: "f8fba656e858a644e3913ff448b5cb4d"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.598860 4428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:06:48.599660 4449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.599838 4428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:48.599946 4428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "967e1aecedbd4d51892507fe9d974c04"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "10fea70e8331f2b7a18776b37e02d0f5"
server_key_iv: "f8fba656e858a644e3913ff448b5cb4d"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.600021 4428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:48.608917 4428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:48.610806 4428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:48.610963 4428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:48.611270 4428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:48.611629 4428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:48.611688 4428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.611730 4428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:48.611758 4428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.619334 4428 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:40209
I20260811 07:06:48.619395 4629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:40209 every 8 connection(s)
I20260811 07:06:48.619719 4428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:48.626840 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4428
I20260811 07:06:48.626950 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:06:48.627164 4018 external_mini_cluster.cc:1428] Setting key 3ad48d24a91bd89d8bad5c995428fadf
I20260811 07:06:48.629563 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.620538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:52191 (local address 127.3.236.190:44979)
0811 07:06:48.620614 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.620616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.620816 (+ 200us) server_negotiation.cc:407] Connection header received
0811 07:06:48.621402 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.621403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.621413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.621478 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.622390 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.622864 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.623942 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.624136 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.624189 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.626256 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.626258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.626258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.626265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.627888 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.628444 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.628446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.628446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.628500 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.628821 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.628822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.628823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.628997 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.629044 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.629418 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.629490 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:06:48.629923 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.620525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44979 (local address 127.3.236.130:52191)
0811 07:06:48.620683 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:48.620725 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:48.621311 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:48.621535 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:48.621539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:48.622121 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:48.622313 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.622318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.622964 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.622964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.623854 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.623856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.624211 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.624212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.624322 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.624789 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:48.624797 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:48.626164 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:48.627977 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.627985 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.627988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.628376 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.628569 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.628569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.628570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.628729 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.629095 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:48.629104 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:48.629279 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 07:06:48.629391 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 07:06:48.629575 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260811 07:06:48.630493 4631 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44979
I20260811 07:06:48.630623 4631 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:48.630836 4631 heartbeater.cc:511] Master 127.3.236.190:44979 requested a full tablet report, sending...
I20260811 07:06:48.631271 4098 ts_manager.cc:194] Registered new tserver with Master: 967e1aecedbd4d51892507fe9d974c04 (127.3.236.130:40209)
I20260811 07:06:48.631676 4098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:52191
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:06:48.655827 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:48.742170 4639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:48.742419 4639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:48.742460 4639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:48.743674 4639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:48.743746 4639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:48.743793 4639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:06:48.745378 4639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:48.745661 4639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:48.745918 4639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:48.751458 4647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:48.751645 4639 server_base.cc:1034] running on GCE node
W20260811 07:06:48.751430 4646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:48.751448 4649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:48.751964 4639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:48.752319 4639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:48.753491 4639 hybrid_clock.cc:648] HybridClock initialized: now 1786432008753464 us; error 43 us; skew 500 ppm
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:48.756157 4639 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:06:48.756625 4639 webserver.cc:460] Webserver started at http://127.3.236.131:44389/ using document root <none> and password file <none>
I20260811 07:06:48.756834 4639 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:48.756872 4639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:48.756960 4639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:48.758059 4639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "889f53ebcc7c4594b4defac26cc96a9a"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "f011e1602f190a8517b8dd296d46ca06"
server_key_iv: "cfbf8a7e371a3c72f31ff47a7c5cb617"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.758450 4639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "889f53ebcc7c4594b4defac26cc96a9a"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "f011e1602f190a8517b8dd296d46ca06"
server_key_iv: "cfbf8a7e371a3c72f31ff47a7c5cb617"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.759938 4639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:06:48.760831 4659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.761162 4639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:06:48.761274 4639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "889f53ebcc7c4594b4defac26cc96a9a"
format_stamp: "Formatted at 2026-08-11 07:06:48 on dist-test-slave-2d19"
server_key: "f011e1602f190a8517b8dd296d46ca06"
server_key_iv: "cfbf8a7e371a3c72f31ff47a7c5cb617"
server_key_version: "encryptionkey@0"
I20260811 07:06:48.761353 4639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:48.781286 4639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:48.783463 4639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:48.783644 4639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:48.784003 4639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:48.784436 4639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:48.784497 4639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.784541 4639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:48.784569 4639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:48.791352 4639 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:43213
I20260811 07:06:48.791392 4841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:43213 every 8 connection(s)
I20260811 07:06:48.791755 4639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:48.801612 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.792663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:39819 (local address 127.3.236.190:44979)
0811 07:06:48.792732 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.792734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.792972 (+ 238us) server_negotiation.cc:407] Connection header received
0811 07:06:48.793467 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.793468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.793480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.793541 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.794596 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.795085 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.796130 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.796372 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.796428 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.798615 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.798617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.798617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.798623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.800416 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.800798 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.800798 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.800799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.800856 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.801084 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.801085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.801085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.801194 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.801258 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.801477 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.801542 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:06:48.801781 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.792645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44979 (local address 127.3.236.131:39819)
0811 07:06:48.792823 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:48.792867 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:48.793375 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:48.793599 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:48.793602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:48.794200 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:48.794397 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.794401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.795191 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.795192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.796047 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:48.796050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.796446 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.796447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:48.796563 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.797115 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:48.797124 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:48.798524 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:48.800501 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.800508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.800510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.800727 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.800920 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:48.800920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:48.800921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:48.801012 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:48.801249 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:48.801252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:48.801359 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 07:06:48.801436 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:06:48.801557 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260811 07:06:48.801872 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4639
I20260811 07:06:48.801936 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:06:48.802081 4018 external_mini_cluster.cc:1428] Setting key da3bcb4a053320af3d92f703476ce02c
I20260811 07:06:48.802608 4842 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44979
I20260811 07:06:48.802716 4842 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:48.802951 4842 heartbeater.cc:511] Master 127.3.236.190:44979 requested a full tablet report, sending...
I20260811 07:06:48.803411 4098 ts_manager.cc:194] Registered new tserver with Master: 889f53ebcc7c4594b4defac26cc96a9a (127.3.236.131:43213)
I20260811 07:06:48.803925 4098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:39819
I20260811 07:06:48.814395 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:48.838169 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.830851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.3.236.129:33119)
0811 07:06:48.831057 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.831063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.831076 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:06:48.831104 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.831106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.831121 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.831189 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.831712 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.832507 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.833210 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.833526 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.833589 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.835663 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.835671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.835671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.835678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.837127 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.837439 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.837442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.837442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.837495 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.837658 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.837659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.837659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.837783 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.837883 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.837944 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.838007 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":118,"threads_started":1}
W20260811 07:06:48.838621 4268 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35958
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:48.862421 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:48.855099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.3.236.129:33119)
0811 07:06:48.855172 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:48.855175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:48.855287 (+ 112us) server_negotiation.cc:407] Connection header received
0811 07:06:48.855388 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:48.855389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:48.855408 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:48.855476 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:48.856062 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.856634 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.857363 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:48.857570 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:48.857688 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:48.860011 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:48.860013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:48.860013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:48.860022 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:48.861285 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.861642 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.861643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.861644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.861700 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:48.861902 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:48.861903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:48.861903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:48.862004 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:48.862146 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:48.862224 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 07:06:48.862309 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:06:48.863024 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4197
I20260811 07:06:48.879392 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4428
I20260811 07:06:48.888821 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4639
I20260811 07:06:48.897487 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4051
2026-08-11T07:06:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2360 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4864](info): set up 2 sockets
Aug 11 07:06:48 dist-test-slave-2d19 krb5kdc[4864](info): commencing operation
krb5kdc: starting...
W20260811 07:06:50.192348 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:06:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:06:50.315147 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35111
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:50.398599 4882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:50.398839 4882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:50.398877 4882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:50.400045 4882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:06:50.400115 4882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:50.400138 4882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:50.400170 4882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:06:50.400216 4882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:06:50.401705 4882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:50.401978 4882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:50.402211 4882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:06:50.407205 4882 server_base.cc:1034] running on GCE node
W20260811 07:06:50.407255 4891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.407255 4889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.407351 4893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:50.407804 4882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:50.408119 4882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:50.409278 4882 hybrid_clock.cc:648] HybridClock initialized: now 1786432010409250 us; error 40 us; skew 500 ppm
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:50.411832 4882 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:06:50.412310 4882 webserver.cc:460] Webserver started at http://127.3.236.190:36837/ using document root <none> and password file <none>
I20260811 07:06:50.412556 4882 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:50.412627 4882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:50.412788 4882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:50.413830 4882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "dff2a16df8324a208473c3f6ce8d3f37"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9f9f13be52fba4706b195f9e17a93775"
server_key_iv: "f9946d9ffe7136760473af42650fbed3"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.414207 4882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "dff2a16df8324a208473c3f6ce8d3f37"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9f9f13be52fba4706b195f9e17a93775"
server_key_iv: "f9946d9ffe7136760473af42650fbed3"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.415552 4882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:06:50.416350 4903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.416572 4882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:50.416702 4882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "dff2a16df8324a208473c3f6ce8d3f37"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9f9f13be52fba4706b195f9e17a93775"
server_key_iv: "f9946d9ffe7136760473af42650fbed3"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.416792 4882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:50.426157 4882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:50.427958 4882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:50.428121 4882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:50.432183 4882 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35111
I20260811 07:06:50.432233 4990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35111 every 8 connection(s)
I20260811 07:06:50.432614 4882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:06:50.433517 4991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:50.435761 4991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37: Bootstrap starting.
I20260811 07:06:50.436471 4991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:50.436784 4991 log.cc:824] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:50.437522 4991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37: No bootstrap required, opened a new log
I20260811 07:06:50.439181 4991 raft_consensus.cc:348] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } }
I20260811 07:06:50.439319 4991 raft_consensus.cc:374] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:50.439342 4991 raft_consensus.cc:729] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff2a16df8324a208473c3f6ce8d3f37, State: Initialized, Role: FOLLOWER
I20260811 07:06:50.439528 4991 consensus_queue.cc:260] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } }
I20260811 07:06:50.439597 4991 raft_consensus.cc:388] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:06:50.439620 4991 raft_consensus.cc:482] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:06:50.439688 4991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:50.440193 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4882
I20260811 07:06:50.440227 4991 raft_consensus.cc:504] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } }
I20260811 07:06:50.440312 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:06:50.440380 4991 leader_election.cc:302] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dff2a16df8324a208473c3f6ce8d3f37; no voters:
I20260811 07:06:50.440491 4018 external_mini_cluster.cc:1428] Setting key b5b5399478d18e5a413375b43d831d5f
I20260811 07:06:50.440565 4991 leader_election.cc:288] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:06:50.440646 4997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:50.440811 4997 raft_consensus.cc:686] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [term 1 LEADER]: Becoming Leader. State: Replica: dff2a16df8324a208473c3f6ce8d3f37, State: Running, Role: LEADER
I20260811 07:06:50.440850 4991 sys_catalog.cc:564] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:06:50.440920 4997 consensus_queue.cc:237] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } }
I20260811 07:06:50.441686 4997 sys_catalog.cc:455] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dff2a16df8324a208473c3f6ce8d3f37. Latest consensus state: current_term: 1 leader_uuid: "dff2a16df8324a208473c3f6ce8d3f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } } }
I20260811 07:06:50.441761 4997 sys_catalog.cc:458] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:50.441944 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dff2a16df8324a208473c3f6ce8d3f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff2a16df8324a208473c3f6ce8d3f37" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35111 } } }
I20260811 07:06:50.442072 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:50.442284 5016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:06:50.442821 5016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:06:50.444345 5016 catalog_manager.cc:1347] Generated new cluster ID: 74de1682c2a74cdf9cda2ec72b818d01
I20260811 07:06:50.444398 5016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:50.453594 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.441158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33860 (local address 127.3.236.190:35111)
0811 07:06:50.441576 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:50.441587 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:50.441618 (+ 31us) server_negotiation.cc:407] Connection header received
0811 07:06:50.441737 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:50.441744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:50.441804 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:50.441943 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:50.442982 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.444430 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.445535 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.446171 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.446289 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:50.449081 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:50.449085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:50.449085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:50.449098 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:50.451841 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.452335 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.452339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.452339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.452443 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.452673 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.452673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.452674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.452962 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:50.453113 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:50.453232 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:06:50.453361 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":71,"threads_started":1}
I20260811 07:06:50.470680 5016 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:06:50.471261 5016 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:06:50.479815 5016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dff2a16df8324a208473c3f6ce8d3f37: Generated new TSK 0
I20260811 07:06:50.480063 5016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:06:50.511270 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:50.595791 5031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:50.596007 5031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:50.596035 5031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:50.597218 5031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:50.597259 5031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:50.597286 5031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:06:50.598785 5031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:50.599037 5031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:50.599283 5031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:50.604418 5039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.604418 5044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.604420 5038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:50.604444 5031 server_base.cc:1034] running on GCE node
I20260811 07:06:50.604815 5031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:50.605077 5031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:50.606252 5031 hybrid_clock.cc:648] HybridClock initialized: now 1786432010606235 us; error 43 us; skew 500 ppm
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:50.608717 5031 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:06:50.609192 5031 webserver.cc:460] Webserver started at http://127.3.236.129:33703/ using document root <none> and password file <none>
I20260811 07:06:50.609441 5031 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:50.609510 5031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:50.609664 5031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:50.610699 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "f7f707cccbcd418aaacb123f8deab9e0"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "60a380e9822ae30d77279a1e0d45665d"
server_key_iv: "acbbcd99dc716fd60e2f021ceb516c50"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.611076 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "f7f707cccbcd418aaacb123f8deab9e0"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "60a380e9822ae30d77279a1e0d45665d"
server_key_iv: "acbbcd99dc716fd60e2f021ceb516c50"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.612445 5031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:06:50.613198 5057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.613369 5031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:50.613461 5031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "f7f707cccbcd418aaacb123f8deab9e0"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "60a380e9822ae30d77279a1e0d45665d"
server_key_iv: "acbbcd99dc716fd60e2f021ceb516c50"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.613545 5031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:50.629938 5031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:50.631939 5031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:50.632135 5031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:50.632479 5031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:50.632827 5031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:50.632885 5031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.632929 5031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:50.632953 5031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.639303 5031 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:41979
I20260811 07:06:50.639355 5238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:41979 every 8 connection(s)
I20260811 07:06:50.639693 5031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:50.647315 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5031
I20260811 07:06:50.647439 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:06:50.647620 4018 external_mini_cluster.cc:1428] Setting key 4a89aac3a800c9275d0db034276f4c77
I20260811 07:06:50.650569 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.640615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:51105 (local address 127.3.236.190:35111)
0811 07:06:50.640678 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:50.640680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:50.640897 (+ 217us) server_negotiation.cc:407] Connection header received
0811 07:06:50.642140 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:50.642142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:50.642158 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:50.642224 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:50.643148 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.643661 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.644740 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.644929 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.644983 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:50.647304 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:50.647305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:50.647305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:50.647311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:50.649166 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.649566 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.649567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.649568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.649636 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.649871 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.649871 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.649872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.650027 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:50.650098 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:50.650390 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 07:06:50.650478 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 07:06:50.650784 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.640561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35111 (local address 127.3.236.129:51105)
0811 07:06:50.640757 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:50.640803 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:50.641981 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:50.642324 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:50.642329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:50.642806 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:50.643047 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:50.643053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.643755 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.643756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:50.644658 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:50.644661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.645008 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.645009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:50.645112 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:50.645721 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:50.645730 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:50.647214 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:50.649251 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:50.649258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:50.649259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:50.649488 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:50.649704 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:50.649705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:50.649705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:50.649807 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:50.650092 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:50.650097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:50.650248 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 07:06:50.650362 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 07:06:50.650521 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":112,"threads_started":1}
I20260811 07:06:50.651284 5241 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35111
I20260811 07:06:50.651381 5241 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:50.651597 5241 heartbeater.cc:511] Master 127.3.236.190:35111 requested a full tablet report, sending...
I20260811 07:06:50.652180 4931 ts_manager.cc:194] Registered new tserver with Master: f7f707cccbcd418aaacb123f8deab9e0 (127.3.236.129:41979)
I20260811 07:06:50.652814 4931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:51105
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:06:50.676132 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:50.761148 5249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:50.761358 5249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:50.761384 5249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:50.762471 5249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:50.762508 5249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:50.762534 5249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:06:50.763947 5249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:50.764127 5249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:50.764379 5249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:50.769526 5265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.769526 5261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.769521 5262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:50.769804 5249 server_base.cc:1034] running on GCE node
I20260811 07:06:50.770094 5249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:50.770354 5249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:50.771517 5249 hybrid_clock.cc:648] HybridClock initialized: now 1786432010771496 us; error 37 us; skew 500 ppm
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:50.774119 5249 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:06:50.774632 5249 webserver.cc:460] Webserver started at http://127.3.236.130:45519/ using document root <none> and password file <none>
I20260811 07:06:50.774871 5249 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:50.774940 5249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:50.775107 5249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:50.776181 5249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "db7dfc5c2858458d940754a413935ef7"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "86625d5ef479fff8b84fe5692a37e635"
server_key_iv: "ffcef0cc9b8068f395efbf172ba9013a"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.776634 5249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "db7dfc5c2858458d940754a413935ef7"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "86625d5ef479fff8b84fe5692a37e635"
server_key_iv: "ffcef0cc9b8068f395efbf172ba9013a"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.777969 5249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:50.778730 5278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.778911 5249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:50.779016 5249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "db7dfc5c2858458d940754a413935ef7"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "86625d5ef479fff8b84fe5692a37e635"
server_key_iv: "ffcef0cc9b8068f395efbf172ba9013a"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.779114 5249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:50.800242 5249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:50.802853 5249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:50.803032 5249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:50.803283 5249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:50.803634 5249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:50.803690 5249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.803732 5249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:50.803759 5249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.810736 5249 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:35627
I20260811 07:06:50.810818 5470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:35627 every 8 connection(s)
I20260811 07:06:50.811172 5249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:06:50.811604 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5249
I20260811 07:06:50.811699 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:06:50.811915 4018 external_mini_cluster.cc:1428] Setting key ac487774de53d5d29265cf43001dcc1f
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
I20260811 07:06:50.821861 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.812048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:33631 (local address 127.3.236.190:35111)
0811 07:06:50.812115 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:50.812117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:50.812394 (+ 277us) server_negotiation.cc:407] Connection header received
0811 07:06:50.813507 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:50.813508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:50.813521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:50.813568 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:50.814203 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.814702 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.815734 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.815933 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.816009 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:50.818767 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:50.818769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:50.818769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:50.818797 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:50.820405 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.820828 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.820829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.820830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.820894 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:50.821154 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:50.821155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:50.821156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:50.821278 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:50.821358 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:50.821679 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 07:06:50.821765 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:06:50.822072 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.812017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35111 (local address 127.3.236.130:33631)
0811 07:06:50.812220 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:50.812292 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:50.813387 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:50.813643 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:50.813648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:50.813912 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:50.814109 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:50.814113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.814800 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.814801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:50.815624 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:50.815626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:50.816029 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:50.816031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:50.816202 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:50.816840 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:50.816849 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:50.818619 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:50.820500 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:50.820508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:50.820510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:50.820745 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:50.820963 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:50.820964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:50.820964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:50.821076 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:50.821349 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:50.821356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:50.821540 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 07:06:50.821656 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 07:06:50.821820 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260811 07:06:50.822748 5472 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35111
I20260811 07:06:50.822871 5472 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:50.823084 5472 heartbeater.cc:511] Master 127.3.236.190:35111 requested a full tablet report, sending...
I20260811 07:06:50.823518 4931 ts_manager.cc:194] Registered new tserver with Master: db7dfc5c2858458d940754a413935ef7 (127.3.236.130:35627)
I20260811 07:06:50.823966 4931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:33631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:06:50.839352 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:50.927771 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:50.928084 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:50.928186 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:50.929454 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:50.929523 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:50.929569 5485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:06:50.931137 5485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:50.931411 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:50.931650 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:50.937067 5493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.937124 5497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:50.937179 5492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:50.937449 5485 server_base.cc:1034] running on GCE node
I20260811 07:06:50.937659 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:50.937901 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:50.939075 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786432010939048 us; error 44 us; skew 500 ppm
Aug 11 07:06:50 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:50.942119 5485 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:06:50.942622 5485 webserver.cc:460] Webserver started at http://127.3.236.131:43107/ using document root <none> and password file <none>
I20260811 07:06:50.942960 5485 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:50.943009 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:50.943158 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:50.944228 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "ff4cdf0c414442eb81eb5a03c6ce0221"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9365f29e1968a1048c5682869eb33b2c"
server_key_iv: "4eeace25ee6b9027d421e6fed3b081b0"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.944680 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "ff4cdf0c414442eb81eb5a03c6ce0221"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9365f29e1968a1048c5682869eb33b2c"
server_key_iv: "4eeace25ee6b9027d421e6fed3b081b0"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.946033 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:50.946995 5514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.947238 5485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:50.947371 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "ff4cdf0c414442eb81eb5a03c6ce0221"
format_stamp: "Formatted at 2026-08-11 07:06:50 on dist-test-slave-2d19"
server_key: "9365f29e1968a1048c5682869eb33b2c"
server_key_iv: "4eeace25ee6b9027d421e6fed3b081b0"
server_key_version: "encryptionkey@0"
I20260811 07:06:50.947465 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:50.987733 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:50.989923 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:50.990109 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:50.990389 5485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:06:50.990751 5485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:50.990809 5485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.990856 5485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:50.990881 5485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:50.998654 5485 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:43605
I20260811 07:06:50.998703 5706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:43605 every 8 connection(s)
I20260811 07:06:50.999042 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:06:51.005860 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5485
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:51.005980 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:06:51.006155 4018 external_mini_cluster.cc:1428] Setting key b94fd8b433428b2ea67ca8acb4991106
I20260811 07:06:51.009613 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.999886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52043 (local address 127.3.236.190:35111)
0811 07:06:50.999933 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:50.999935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.000200 (+ 265us) server_negotiation.cc:407] Connection header received
0811 07:06:51.001291 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.001293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.001304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.001359 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:51.002076 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.002563 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.003612 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.003811 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.003863 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.006527 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:51.006528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:51.006529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:51.006545 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:51.008099 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.008573 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.008575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.008575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.008646 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.008919 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.008920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.008920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.009042 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:51.009119 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.009416 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.009509 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:06:51.009863 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:50.999849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35111 (local address 127.3.236.131:52043)
0811 07:06:51.000034 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.000078 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.001177 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.001423 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.001426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.001741 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:51.001995 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.002001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.002665 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.002665 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.003523 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.003526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.003912 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.003913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.004047 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.004663 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:51.004672 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:51.006415 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:51.008205 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:51.008213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:51.008216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:51.008492 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:51.008717 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:51.008719 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:51.008719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:51.008843 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:51.009113 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:51.009120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:51.009273 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.009379 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.009548 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260811 07:06:51.010454 5707 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35111
I20260811 07:06:51.010576 5707 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:51.010769 5707 heartbeater.cc:511] Master 127.3.236.190:35111 requested a full tablet report, sending...
I20260811 07:06:51.011191 4931 ts_manager.cc:194] Registered new tserver with Master: ff4cdf0c414442eb81eb5a03c6ce0221 (127.3.236.131:43605)
I20260811 07:06:51.011792 4931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:52043
I20260811 07:06:51.017980 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:06:51.025672 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.019963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.3.236.190:35111)
0811 07:06:51.020023 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.020025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.020154 (+ 129us) server_negotiation.cc:407] Connection header received
0811 07:06:51.020226 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.020227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.020236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.020338 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:51.020884 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.021358 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.022055 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.022265 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.022317 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.023413 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:51.023414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:51.023415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:51.023421 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:51.024717 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.025024 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.025025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.025025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.025092 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.025258 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.025258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.025258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.025393 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:51.025467 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.025551 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.025609 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 07:06:51.028151 4931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:51.029896 4931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:51.038589 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.3.236.130:35627)
0811 07:06:51.034831 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.034835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.034844 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:06:51.034881 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.034883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.034897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.034987 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:51.035124 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036305 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.037437 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038150 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.038239 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.038302 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.038465 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.038504 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260811 07:06:51.038576 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:35627 (local address 127.0.0.1:55726)
0811 07:06:51.034365 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.034407 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.034473 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.034999 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.035000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.035003 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:51.035054 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.035056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036419 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.036420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.037335 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.037339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038238 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.038239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.038321 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.038337 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.038432 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.038492 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":92,"thread_start_us":107,"threads_started":1}
I20260811 07:06:51.038920 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41979 (local address 127.0.0.1:60308)
0811 07:06:51.034375 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.034410 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.034480 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.034845 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.034849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.034853 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:51.034930 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.034932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036423 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.036424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.037730 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.037733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038666 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.038667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.038739 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.038743 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.038775 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.038824 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":55,"thread_start_us":67,"threads_started":1}
I20260811 07:06:51.038920 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:43605 (local address 127.0.0.1:41320)
0811 07:06:51.034704 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.034714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.034762 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.035207 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.035208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.035210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:51.035235 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.035236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036355 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.036356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.037734 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.037735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038712 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.038713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.038764 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.038768 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.038814 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.038841 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":101,"thread_start_us":93,"threads_started":1}
I20260811 07:06:51.039119 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.3.236.131:43605)
0811 07:06:51.034938 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.034942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.034970 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:06:51.035011 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.035013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.035028 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.035121 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:51.035312 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036199 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.037839 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038554 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.038624 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.038683 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.038963 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.039013 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":132,"threads_started":1}
I20260811 07:06:51.039412 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.034349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.3.236.129:41979)
0811 07:06:51.034647 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.034652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.034663 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:06:51.034707 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.034709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.034728 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.035011 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:51.035118 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.036282 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.037839 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.038550 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.039151 (+ 601us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.039230 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.039258 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.039338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":121,"threads_started":1}
I20260811 07:06:51.039961 5600 tablet_service.cc:1467] Processing CreateTablet for tablet c8aebfc94deb49bd9ebb32af9ec6e64f (DEFAULT_TABLE table=test-table [id=f018a0e04dc947b1a047005e6b85c4ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:51.039969 5357 tablet_service.cc:1467] Processing CreateTablet for tablet c8aebfc94deb49bd9ebb32af9ec6e64f (DEFAULT_TABLE table=test-table [id=f018a0e04dc947b1a047005e6b85c4ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:51.040077 5126 tablet_service.cc:1467] Processing CreateTablet for tablet c8aebfc94deb49bd9ebb32af9ec6e64f (DEFAULT_TABLE table=test-table [id=f018a0e04dc947b1a047005e6b85c4ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:51.040370 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aebfc94deb49bd9ebb32af9ec6e64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:51.040373 5357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aebfc94deb49bd9ebb32af9ec6e64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:51.040408 5126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aebfc94deb49bd9ebb32af9ec6e64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:51.043322 5744 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Bootstrap starting.
I20260811 07:06:51.043831 5746 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Bootstrap starting.
I20260811 07:06:51.044059 5747 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Bootstrap starting.
I20260811 07:06:51.044514 5744 tablet_bootstrap.cc:654] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:51.044560 5746 tablet_bootstrap.cc:654] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:51.044760 5747 tablet_bootstrap.cc:654] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:51.045256 5746 log.cc:824] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:51.045404 5747 log.cc:824] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:51.046135 5746 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: No bootstrap required, opened a new log
I20260811 07:06:51.046217 5746 ts_tablet_manager.cc:1397] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:06:51.046260 5747 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: No bootstrap required, opened a new log
I20260811 07:06:51.046330 5747 ts_tablet_manager.cc:1397] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:51.047293 5744 log.cc:824] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:51.048086 5744 tablet_bootstrap.cc:492] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: No bootstrap required, opened a new log
I20260811 07:06:51.048172 5744 ts_tablet_manager.cc:1397] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:06:51.048419 5746 raft_consensus.cc:348] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.048488 5747 raft_consensus.cc:348] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.048625 5746 raft_consensus.cc:374] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:51.048625 5747 raft_consensus.cc:374] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:51.048664 5747 raft_consensus.cc:729] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7dfc5c2858458d940754a413935ef7, State: Initialized, Role: FOLLOWER
I20260811 07:06:51.048664 5746 raft_consensus.cc:729] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f707cccbcd418aaacb123f8deab9e0, State: Initialized, Role: FOLLOWER
I20260811 07:06:51.048784 5746 consensus_queue.cc:260] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.048810 5747 consensus_queue.cc:260] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.049093 5746 ts_tablet_manager.cc:1428] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 07:06:51.049104 5747 ts_tablet_manager.cc:1428] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 07:06:51.049259 5241 heartbeater.cc:503] Master 127.3.236.190:35111 was elected leader, sending a full tablet report...
I20260811 07:06:51.049549 5472 heartbeater.cc:503] Master 127.3.236.190:35111 was elected leader, sending a full tablet report...
I20260811 07:06:51.051169 5744 raft_consensus.cc:348] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.051307 5744 raft_consensus.cc:374] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:51.051330 5744 raft_consensus.cc:729] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff4cdf0c414442eb81eb5a03c6ce0221, State: Initialized, Role: FOLLOWER
I20260811 07:06:51.051445 5744 consensus_queue.cc:260] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.051739 5744 ts_tablet_manager.cc:1428] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 07:06:51.051769 5707 heartbeater.cc:503] Master 127.3.236.190:35111 was elected leader, sending a full tablet report...
W20260811 07:06:51.061879 5475 tablet.cc:2369] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:06:51.140452 5243 tablet.cc:2369] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:51.169047 5760 raft_consensus.cc:482] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:51.169225 5760 raft_consensus.cc:504] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.169737 5760 leader_election.cc:288] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db7dfc5c2858458d940754a413935ef7 (127.3.236.130:35627), f7f707cccbcd418aaacb123f8deab9e0 (127.3.236.129:41979)
I20260811 07:06:51.172691 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.170047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:35603 (local address 127.3.236.129:41979)
0811 07:06:51.170106 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.170108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.170121 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:06:51.170180 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.170181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.170190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.170234 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:51.170479 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.170938 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.171899 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.172319 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.172400 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.172410 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.172572 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.172610 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:06:51.172705 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.170013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:35569 (local address 127.3.236.130:35627)
0811 07:06:51.170079 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.170081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.170256 (+ 175us) server_negotiation.cc:407] Connection header received
0811 07:06:51.170320 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.170321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.170332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.170373 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:51.170545 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.170993 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.171952 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.172342 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.172420 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.172429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.172595 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.172633 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:06:51.172719 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.169969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41979 (local address 127.3.236.131:35603)
0811 07:06:51.170035 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.170047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.170097 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.170294 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.170295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.170297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:51.170403 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.170405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.171032 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.171034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.171790 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.171792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.172399 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.172400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.172485 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.172487 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.172564 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.172611 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:06:51.172725 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.169977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:35627 (local address 127.3.236.131:35569)
0811 07:06:51.170178 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:51.170186 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:51.170262 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:51.170399 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:51.170399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:51.170401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:51.170463 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.170464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.171086 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.171087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.171834 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:51.171836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.172432 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.172433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:51.172504 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.172508 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:51.172585 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:06:51.172627 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":19,"thread_start_us":96,"threads_started":1}
I20260811 07:06:51.173236 5164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aebfc94deb49bd9ebb32af9ec6e64f" candidate_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f707cccbcd418aaacb123f8deab9e0" is_pre_election: true
I20260811 07:06:51.173377 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aebfc94deb49bd9ebb32af9ec6e64f" candidate_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db7dfc5c2858458d940754a413935ef7" is_pre_election: true
I20260811 07:06:51.173455 5164 raft_consensus.cc:2454] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff4cdf0c414442eb81eb5a03c6ce0221 in term 0.
I20260811 07:06:51.173542 5409 raft_consensus.cc:2454] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff4cdf0c414442eb81eb5a03c6ce0221 in term 0.
I20260811 07:06:51.173724 5515 leader_election.cc:302] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f7f707cccbcd418aaacb123f8deab9e0, ff4cdf0c414442eb81eb5a03c6ce0221; no voters:
I20260811 07:06:51.173904 5760 raft_consensus.cc:2781] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:51.173944 5760 raft_consensus.cc:482] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:51.173960 5760 raft_consensus.cc:3037] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:51.174652 5760 raft_consensus.cc:504] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.174793 5760 leader_election.cc:288] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [CANDIDATE]: Term 1 election: Requested vote from peers db7dfc5c2858458d940754a413935ef7 (127.3.236.130:35627), f7f707cccbcd418aaacb123f8deab9e0 (127.3.236.129:41979)
I20260811 07:06:51.174938 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aebfc94deb49bd9ebb32af9ec6e64f" candidate_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db7dfc5c2858458d940754a413935ef7"
I20260811 07:06:51.175086 5409 raft_consensus.cc:3037] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:51.175879 5409 raft_consensus.cc:2454] T c8aebfc94deb49bd9ebb32af9ec6e64f P db7dfc5c2858458d940754a413935ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff4cdf0c414442eb81eb5a03c6ce0221 in term 1.
I20260811 07:06:51.176023 5164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aebfc94deb49bd9ebb32af9ec6e64f" candidate_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f707cccbcd418aaacb123f8deab9e0"
I20260811 07:06:51.176074 5519 leader_election.cc:302] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db7dfc5c2858458d940754a413935ef7, ff4cdf0c414442eb81eb5a03c6ce0221; no voters:
I20260811 07:06:51.176127 5164 raft_consensus.cc:3037] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:51.176180 5760 raft_consensus.cc:2781] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:51.176397 5760 raft_consensus.cc:686] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [term 1 LEADER]: Becoming Leader. State: Replica: ff4cdf0c414442eb81eb5a03c6ce0221, State: Running, Role: LEADER
I20260811 07:06:51.176501 5760 consensus_queue.cc:237] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } }
I20260811 07:06:51.177111 5164 raft_consensus.cc:2454] T c8aebfc94deb49bd9ebb32af9ec6e64f P f7f707cccbcd418aaacb123f8deab9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff4cdf0c414442eb81eb5a03c6ce0221 in term 1.
I20260811 07:06:51.177428 4927 catalog_manager.cc:5495] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff4cdf0c414442eb81eb5a03c6ce0221 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db7dfc5c2858458d940754a413935ef7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7f707cccbcd418aaacb123f8deab9e0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff4cdf0c414442eb81eb5a03c6ce0221" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43605 } health_report { overall_health: HEALTHY } } }
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:51.221730 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.213955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.3.236.129:41979)
0811 07:06:51.214050 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.214052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.214099 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:06:51.214165 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.214166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.214175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.214226 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:51.214765 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.215410 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.216199 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.216509 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.216566 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.218976 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:51.218988 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:51.218988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:51.218995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:51.220745 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.221053 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.221056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.221057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.221152 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.221319 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.221319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.221320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.221449 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:51.221572 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.221601 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.221662 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:51.248677 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.241410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.3.236.129:41979)
0811 07:06:51.241477 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.241479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.241577 (+ 98us) server_negotiation.cc:407] Connection header received
0811 07:06:51.241649 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.241650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.241661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.241721 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:51.242471 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.242975 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.243716 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.243935 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.244013 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.246275 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:51.246277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:51.246277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:51.246284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:51.247432 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.247796 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.247798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.247798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.247859 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.248064 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.248065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.248065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.248167 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:51.248287 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.248478 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.248560 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 07:06:51.249751 5709 tablet.cc:2369] T c8aebfc94deb49bd9ebb32af9ec6e64f P ff4cdf0c414442eb81eb5a03c6ce0221: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[4864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:51.271967 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:51.265566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.3.236.129:41979)
0811 07:06:51.265666 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:51.265668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:51.265699 (+ 31us) server_negotiation.cc:407] Connection header received
0811 07:06:51.265802 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:51.265803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:51.265813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:51.265866 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:51.266388 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.266849 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.267530 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:51.267703 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:51.267752 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:51.269874 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:51.269876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:51.269876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:51.269882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:51.271041 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.271342 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.271343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.271343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.271392 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:51.271552 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:51.271553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:51.271553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:51.271647 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:51.271721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:51.271828 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 07:06:51.271899 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 07:06:51.272629 5097 tablet_service.cc:2908] Rejecting scan request for tablet c8aebfc94deb49bd9ebb32af9ec6e64f: Uninitialized: clean time has not yet been initialized
I20260811 07:06:51.273669 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5031
I20260811 07:06:51.286919 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5249
I20260811 07:06:51.292440 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5485
I20260811 07:06:51.304491 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4882
2026-08-11T07:06:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2409 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[5788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[5788](info): set up 2 sockets
Aug 11 07:06:51 dist-test-slave-2d19 krb5kdc[5788](info): commencing operation
krb5kdc: starting...
W20260811 07:06:52.640686 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 07:06:52.845036 4018 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.204s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 07:06:53.035080 4018 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.190s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 07:06:53.145668 4018 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.110s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/test-user.keytab.
W20260811 07:06:53.556198 4018 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.410s user 0.000s sys 0.002s
Aug 11 07:06:53 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:06:53.602941 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:39805
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:39805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:53.685420 5805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:53.685635 5805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:53.685657 5805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:53.686729 5805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:06:53.686766 5805 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:06:53.686779 5805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:53.686793 5805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:53.686805 5805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:06:53.686817 5805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:06:53.688205 5805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:39805
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:39805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:53.688498 5805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:53.688757 5805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:53.693864 5818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:53.693871 5816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:53.693904 5805 server_base.cc:1034] running on GCE node
W20260811 07:06:53.693881 5820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:53.694424 5805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:53.694751 5805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:53.695928 5805 hybrid_clock.cc:648] HybridClock initialized: now 1786432013695903 us; error 35 us; skew 500 ppm
Aug 11 07:06:53 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:53.698622 5805 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:06:53.699177 5805 webserver.cc:460] Webserver started at http://127.3.236.190:44809/ using document root <none> and password file <none>
I20260811 07:06:53.699469 5805 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:53.699535 5805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:53.699681 5805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:53.700884 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "91642af5af694d6f85197e23723903b5"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "4642a7a8df369275b916b89a2bf59eeb"
server_key_iv: "4a9092d049d3f8bc90d08acac961c013"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.701267 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "91642af5af694d6f85197e23723903b5"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "4642a7a8df369275b916b89a2bf59eeb"
server_key_iv: "4a9092d049d3f8bc90d08acac961c013"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.702630 5805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 07:06:53.703373 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:53.703563 5805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:53.703673 5805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "91642af5af694d6f85197e23723903b5"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "4642a7a8df369275b916b89a2bf59eeb"
server_key_iv: "4a9092d049d3f8bc90d08acac961c013"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.703752 5805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:53.713651 5805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:53.715826 5805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:53.716029 5805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:53.721549 5805 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:39805
I20260811 07:06:53.721606 5946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:39805 every 8 connection(s)
I20260811 07:06:53.721977 5805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:06:53.722654 5947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:53.725085 5947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Bootstrap starting.
I20260811 07:06:53.725703 5947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:53.726022 5947 log.cc:824] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:53.726732 5947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: No bootstrap required, opened a new log
I20260811 07:06:53.727835 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5805
I20260811 07:06:53.727949 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:06:53.728135 4018 external_mini_cluster.cc:1428] Setting key 6c688d82f51cb85f933c92b001dfb4c1
I20260811 07:06:53.729573 5947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } }
I20260811 07:06:53.729754 5947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:53.729840 5947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91642af5af694d6f85197e23723903b5, State: Initialized, Role: FOLLOWER
I20260811 07:06:53.730003 5947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } }
I20260811 07:06:53.730090 5947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:06:53.730114 5947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:06:53.730175 5947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:53.730870 5947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } }
I20260811 07:06:53.730985 5947 leader_election.cc:302] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91642af5af694d6f85197e23723903b5; no voters:
I20260811 07:06:53.731158 5947 leader_election.cc:288] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:06:53.731223 5956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:53.731369 5956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [term 1 LEADER]: Becoming Leader. State: Replica: 91642af5af694d6f85197e23723903b5, State: Running, Role: LEADER
I20260811 07:06:53.731406 5947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:06:53.732362 5956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } }
I20260811 07:06:53.733222 5957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91642af5af694d6f85197e23723903b5. Latest consensus state: current_term: 1 leader_uuid: "91642af5af694d6f85197e23723903b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } } }
W20260811 07:06:53.733630 5972 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 07:06:53.734010 5957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [sys.catalog]: This master's current role is: LEADER
W20260811 07:06:53.734021 5972 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:06:53.734288 5975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:06:53.733945 5958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91642af5af694d6f85197e23723903b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91642af5af694d6f85197e23723903b5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39805 } } }
I20260811 07:06:53.734426 5958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:53.734555 5975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 07:06:53 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:53.736714 5975 catalog_manager.cc:1347] Generated new cluster ID: ea126cc3bd4944fa98c523ca498d58cd
I20260811 07:06:53.736779 5975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:06:53.741175 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:53.728776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.3.236.190:39805)
0811 07:06:53.729012 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:53.729023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:53.729076 (+ 53us) server_negotiation.cc:407] Connection header received
0811 07:06:53.729521 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:53.729530 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:53.729595 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:53.729752 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:53.730540 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.731950 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.733070 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.733637 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.733774 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:53.736675 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:53.736680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:53.736680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:53.736692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:53.739426 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.739886 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.739889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.739893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.740005 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.740232 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.740233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.740233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.740557 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:53.740707 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:53.740836 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 07:06:53.740969 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260811 07:06:53.752404 5975 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:06:53.752939 5975 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:06:53.755147 5975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Generated new TSK 0
I20260811 07:06:53.755333 5975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:06:53.791287 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:39805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:53.875244 5986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:53.875453 5986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:53.875479 5986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:53.876741 5986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:53.876783 5986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:53.876804 5986 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:53.876847 5986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:06:53.878301 5986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:53.878567 5986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:53.878808 5986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:53.883956 5998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:53.883956 5993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:53.884019 5994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:53.884089 5986 server_base.cc:1034] running on GCE node
I20260811 07:06:53.884580 5986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:53.884843 5986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:53.885993 5986 hybrid_clock.cc:648] HybridClock initialized: now 1786432013885984 us; error 32 us; skew 500 ppm
Aug 11 07:06:53 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:53.888948 5986 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:06:53.889536 5986 webserver.cc:460] Webserver started at http://127.3.236.129:42595/ using document root <none> and password file <none>
I20260811 07:06:53.889763 5986 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:53.889811 5986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:53.889918 5986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:53.891151 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "160565cc916b47d2b015727a7096b568"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "b996cd6d30d800bbfaf9b9d440997432"
server_key_iv: "e1b4b0a3e89f0490496846346d412720"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.891500 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "160565cc916b47d2b015727a7096b568"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "b996cd6d30d800bbfaf9b9d440997432"
server_key_iv: "e1b4b0a3e89f0490496846346d412720"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.893072 5986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:06:53.894057 6014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:53.894272 5986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:53.894387 5986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "160565cc916b47d2b015727a7096b568"
format_stamp: "Formatted at 2026-08-11 07:06:53 on dist-test-slave-2d19"
server_key: "b996cd6d30d800bbfaf9b9d440997432"
server_key_iv: "e1b4b0a3e89f0490496846346d412720"
server_key_version: "encryptionkey@0"
I20260811 07:06:53.894467 5986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:53.912345 5986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:53.914222 5986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:53.914387 5986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:53.914959 5986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:53.915005 5986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:53.915030 5986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:53.915042 5986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:06:53 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:53.922920 5986 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:35699
I20260811 07:06:53.923281 5986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:06:53.923255 6213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:35699 every 8 connection(s)
I20260811 07:06:53.925771 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:53.915527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:48491 (local address 127.3.236.190:39805)
0811 07:06:53.915584 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:53.915586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:53.915832 (+ 246us) server_negotiation.cc:407] Connection header received
0811 07:06:53.916546 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:53.916547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:53.916565 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:53.916623 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:53.917612 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.918343 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.919349 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.919572 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.919646 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:53.922605 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:53.922607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:53.922607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:53.922617 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:53.924329 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.924768 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.924770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.924770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.924832 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.925109 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.925110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.925111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.925233 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:53.925306 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:53.925590 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 07:06:53.925680 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:06:53.925947 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:53.915530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.129:48491)
0811 07:06:53.915706 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:53.915742 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:53.916327 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:53.916756 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:53.916760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:53.917149 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:53.917442 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:53.917449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.918442 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.918443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:53.919252 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:53.919255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.919682 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.919684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:53.919837 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:53.920446 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:53.920456 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:53.922504 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:53.924431 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:53.924438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:53.924440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:53.924671 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:53.924905 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:53.924906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:53.924906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:53.925031 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:53.925311 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:53.925318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:53.925453 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 07:06:53.925555 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 07:06:53.925719 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260811 07:06:53.926493 6214 heartbeater.cc:347] Connected to a master server at 127.3.236.190:39805
I20260811 07:06:53.926580 6214 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:53.926788 6214 heartbeater.cc:511] Master 127.3.236.190:39805 requested a full tablet report, sending...
I20260811 07:06:53.926930 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5986
I20260811 07:06:53.927034 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:06:53.927244 4018 external_mini_cluster.cc:1428] Setting key 93bce7471af22a91d0d393fe6ab35e18
I20260811 07:06:53.927345 5859 ts_manager.cc:194] Registered new tserver with Master: 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699)
I20260811 07:06:53.928145 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:48491
I20260811 07:06:53.934477 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:53.927814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:53315 (local address 127.3.236.190:39805)
0811 07:06:53.928363 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:53.928365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:53.928373 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:06:53.928412 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:53.928413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:53.928424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:53.928487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:53.928932 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.929414 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.930230 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.930528 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.930614 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:53.931547 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:53.931548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:53.931549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:53.931556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:53.933319 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.933751 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.933752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.933753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.933809 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:53.934029 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:53.934030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:53.934030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:53.934127 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:53.934180 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:53.934354 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 07:06:53.934418 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530}
I20260811 07:06:53.934571 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:53.927696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.129:53315)
0811 07:06:53.927878 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:53.927891 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:53.927941 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:53.928497 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:53.928499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:53.928759 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:53.928840 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:53.928842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.929524 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.929525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:53.930147 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:53.930149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:53.930609 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:53.930610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:53.930667 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:53.931104 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:53.931109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:53.931470 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:53.933403 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:53.933405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:53.933405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:53.933662 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:53.933882 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:53.933883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:53.933884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:53.933961 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:53.934195 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:53.934196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:53.934253 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 07:06:53.934340 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:06:53.934449 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:06:53.955637 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:39805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:54.040714 6231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:54.040925 6231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:54.040949 6231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:54.042052 6231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:54.042088 6231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:54.042114 6231 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:54.042131 6231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:06:54.043478 6231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:54.043655 6231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:54.043828 6231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:54.048913 6240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:54.048928 6245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:54.048921 6241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:54.048985 6231 server_base.cc:1034] running on GCE node
I20260811 07:06:54.049328 6231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:54.049603 6231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:54.050769 6231 hybrid_clock.cc:648] HybridClock initialized: now 1786432014050757 us; error 30 us; skew 500 ppm
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:54.053337 6231 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:06:54.053786 6231 webserver.cc:460] Webserver started at http://127.3.236.130:44633/ using document root <none> and password file <none>
I20260811 07:06:54.053972 6231 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:54.054011 6231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:54.054108 6231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:54.055087 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "ffcd32232f6f4128b07ea490b61793d2"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "174ab358bcba6ee4557f14e3bd436cf7"
server_key_iv: "7977e566210507ceed1a469e213fa086"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.055404 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "ffcd32232f6f4128b07ea490b61793d2"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "174ab358bcba6ee4557f14e3bd436cf7"
server_key_iv: "7977e566210507ceed1a469e213fa086"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.056742 6231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:06:54.057525 6261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:54.057746 6231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:54.057827 6231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "ffcd32232f6f4128b07ea490b61793d2"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "174ab358bcba6ee4557f14e3bd436cf7"
server_key_iv: "7977e566210507ceed1a469e213fa086"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.057883 6231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:54.079558 6231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:54.081794 6231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:54.081910 6231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:54.082494 6231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:54.082561 6231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:54.082597 6231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:54.082623 6231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:54.090554 6231 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41575
I20260811 07:06:54.090611 6468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41575 every 8 connection(s)
I20260811 07:06:54.091015 6231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:06:54.091136 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6231
I20260811 07:06:54.091241 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:06:54.091454 4018 external_mini_cluster.cc:1428] Setting key 3d609972969044ce7f553ec9976946dd
I20260811 07:06:54.093276 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.082874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:35739 (local address 127.3.236.190:39805)
0811 07:06:54.082917 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.082919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.083177 (+ 258us) server_negotiation.cc:407] Connection header received
0811 07:06:54.083693 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.083695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.083710 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.083797 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.084780 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.085623 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.086734 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.086956 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.087038 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.089777 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.089779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.089779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.089788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.091649 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.092203 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.092205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.092205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.092320 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.092589 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.092590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.092590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.092720 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.092795 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.093088 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.093183 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:06:54.093492 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.082871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.130:35739)
0811 07:06:54.083043 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.083089 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.083574 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.083851 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.083855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.084301 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:54.084522 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.084527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.085736 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.085737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.086628 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.086631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.087060 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.087061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.087207 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.087788 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:54.087800 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:54.089647 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:54.091824 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.091832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.091834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.092113 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.092390 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.092391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.092391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.092514 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.092798 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:54.092806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:54.092953 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.093067 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.093236 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260811 07:06:54.095166 6469 heartbeater.cc:347] Connected to a master server at 127.3.236.190:39805
I20260811 07:06:54.095260 6469 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:54.095453 6469 heartbeater.cc:511] Master 127.3.236.190:39805 requested a full tablet report, sending...
I20260811 07:06:54.095882 5859 ts_manager.cc:194] Registered new tserver with Master: ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130:41575)
I20260811 07:06:54.096338 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:35739
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
I20260811 07:06:54.102165 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.095453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:34385 (local address 127.3.236.190:39805)
0811 07:06:54.095493 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.095495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.095504 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:06:54.095526 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.095527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.095537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.095599 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.096005 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.096832 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.098055 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.098247 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.098300 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.099459 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.099461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.099461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.099467 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.100903 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.101334 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.101335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.101335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.101391 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.101617 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.101618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.101618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.101816 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.101881 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.102040 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.102103 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:06:54.102201 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.095263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.130:34385)
0811 07:06:54.095404 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.095417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.095472 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.095603 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.095604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.095834 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:54.095909 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.095911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.096957 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.096958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.097961 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.097965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.098329 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.098330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.098426 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.098878 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:54.098884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:54.099353 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:54.100991 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.100993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.100993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.101231 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.101470 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.101471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.101471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.101540 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.101880 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:54.101881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:54.101936 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.102023 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.102128 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:06:54.119573 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:39805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:54.207317 6480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:54.207535 6480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:54.207558 6480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:54.208796 6480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:54.208837 6480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:54.208858 6480 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:54.208897 6480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:06:54.210310 6480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:54.210563 6480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:54.210793 6480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:54.215821 6502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:54.215869 6492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:54.215838 6498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:54.216099 6480 server_base.cc:1034] running on GCE node
I20260811 07:06:54.216325 6480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:54.216609 6480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:54.217775 6480 hybrid_clock.cc:648] HybridClock initialized: now 1786432014217765 us; error 24 us; skew 500 ppm
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:54.220407 6480 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:06:54.220880 6480 webserver.cc:460] Webserver started at http://127.3.236.131:32997/ using document root <none> and password file <none>
I20260811 07:06:54.221138 6480 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:54.221241 6480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:54.221408 6480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:54.222601 6480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "083ee4009bcc4711a0b4907a97f5962f"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "810bec6bbd1814e7c81ce812a2f92a5b"
server_key_iv: "b7d5ee86b07349ee991a2cb16ada19a5"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.222997 6480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "083ee4009bcc4711a0b4907a97f5962f"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "810bec6bbd1814e7c81ce812a2f92a5b"
server_key_iv: "b7d5ee86b07349ee991a2cb16ada19a5"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.224587 6480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:06:54.225418 6518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:54.225657 6480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:54.225777 6480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "083ee4009bcc4711a0b4907a97f5962f"
format_stamp: "Formatted at 2026-08-11 07:06:54 on dist-test-slave-2d19"
server_key: "810bec6bbd1814e7c81ce812a2f92a5b"
server_key_iv: "b7d5ee86b07349ee991a2cb16ada19a5"
server_key_version: "encryptionkey@0"
I20260811 07:06:54.225860 6480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:54.235145 6480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:54.237319 6480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:54.237493 6480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:54.238040 6480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:54.238114 6480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:54.238166 6480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:54.238193 6480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:54.246344 6480 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35429
I20260811 07:06:54.246416 6725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35429 every 8 connection(s)
I20260811 07:06:54.246807 6480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:06:54.249661 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.238602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:54999 (local address 127.3.236.190:39805)
0811 07:06:54.238678 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.238681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.238952 (+ 271us) server_negotiation.cc:407] Connection header received
0811 07:06:54.239478 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.239479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.239492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.239543 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.240368 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.240870 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.242396 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.242712 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.242783 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.246591 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.246593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.246593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.246602 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.248191 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.248647 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.248648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.248649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.248714 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.248970 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.248971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.248971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.249098 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.249175 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.249481 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.249565 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:06:54.249859 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.238589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.131:54999)
0811 07:06:54.238845 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.238873 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.239365 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.239616 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.239619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.239914 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:54.240227 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.240234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.240972 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.240973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.242303 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.242307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.242814 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.242816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.243006 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.243588 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:54.243609 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:54.246486 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:54.248324 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.248332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.248334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.248562 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.248787 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.248788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.248789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.248897 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.249167 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:54.249174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:54.249306 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.249456 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.249626 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":99,"threads_started":1}
I20260811 07:06:54.250373 6728 heartbeater.cc:347] Connected to a master server at 127.3.236.190:39805
I20260811 07:06:54.250458 6728 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:54.250645 6728 heartbeater.cc:511] Master 127.3.236.190:39805 requested a full tablet report, sending...
I20260811 07:06:54.251091 5859 ts_manager.cc:194] Registered new tserver with Master: 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429)
I20260811 07:06:54.251583 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:54999
I20260811 07:06:54.254827 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6480
I20260811 07:06:54.254937 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:06:54.255121 4018 external_mini_cluster.cc:1428] Setting key ab21c64197323ecde236c23888d30071
I20260811 07:06:54.256040 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:06:54.257615 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.251521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:60011 (local address 127.3.236.190:39805)
0811 07:06:54.251567 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.251569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.251646 (+ 77us) server_negotiation.cc:407] Connection header received
0811 07:06:54.251778 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.251779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.251794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.251837 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.252211 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.252812 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.253488 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.253681 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.253738 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.254537 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.254538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.254538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.254544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.256386 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.256746 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.256747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.256747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.256815 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.257049 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.257051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.257051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.257167 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.257244 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.257424 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.257519 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:06:54.257625 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.251426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.3.236.131:60011)
0811 07:06:54.251581 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.251593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.251661 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.251863 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.251864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.252071 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:54.252128 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.252130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.252901 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.252902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.253411 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.253412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.253753 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.253753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.253811 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.254188 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:54.254192 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:54.254457 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:54.256472 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.256473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.256474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.256665 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.256892 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.256893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.256893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.256961 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.257248 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:54.257249 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:54.257300 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.257393 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.257507 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:54.281504 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.274217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.3.236.190:39805)
0811 07:06:54.274287 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.274289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.274300 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:06:54.274369 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.274370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.274387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.274458 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.275044 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.275574 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.276507 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.276713 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.276793 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.279027 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.279029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.279029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.279035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.280512 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.280872 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.280873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.280873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.280926 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.281103 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.281103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.281104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.281203 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.281276 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.281383 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.281441 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:06:54.283177 5859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:54.284929 5859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:54.292037 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.3.236.130:41575)
0811 07:06:54.288045 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.288050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.288061 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:06:54.288104 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.288106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.288123 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.288208 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.288464 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289286 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291017 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291560 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291660 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.291728 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.291866 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.291915 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
I20260811 07:06:54.292039 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41575 (local address 127.0.0.1:36284)
0811 07:06:54.287826 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.287835 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.287882 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.288308 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.288309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.288311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.288371 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.288373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289385 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.289386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.290952 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.290954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291660 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.291747 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.291763 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.291841 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.291915 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":32,"thread_start_us":73,"threads_started":1}
I20260811 07:06:54.292312 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:35699 (local address 127.0.0.1:35424)
0811 07:06:54.287618 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.287669 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.287742 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.288159 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.288162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.288164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.288235 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.288238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289633 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.289634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.290848 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.290851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291725 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.291867 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.291874 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.291920 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.291968 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":7,"thread_start_us":49,"threads_started":1}
I20260811 07:06:54.292357 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.3.236.129:35699)
0811 07:06:54.287976 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.287980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.287988 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:06:54.288047 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.288049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.288072 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.288153 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.288350 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289509 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.290923 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291629 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.292089 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.292167 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.292195 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.292242 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":118,"threads_started":1}
I20260811 07:06:54.292474 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35429 (local address 127.0.0.1:56460)
0811 07:06:54.287715 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.287728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.287782 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.288479 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.288480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.288483 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.288535 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.288536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289728 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.289729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.290758 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.290760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291777 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291777 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.291851 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.291855 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.291924 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.291968 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260811 07:06:54.292894 6352 tablet_service.cc:1467] Processing CreateTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f (DEFAULT_TABLE table=test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.293013 6108 tablet_service.cc:1467] Processing CreateTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f (DEFAULT_TABLE table=test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.293227 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.287535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.3.236.131:35429)
0811 07:06:54.288320 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.288324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.288336 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:06:54.288382 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.288384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.288402 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.288481 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.288620 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.289632 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.290848 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.291605 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.291686 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.291747 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.292997 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.293050 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":114,"threads_started":1}
I20260811 07:06:54.293295 6352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ce7eb588e44c3abcc6a0d1e12b784f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.293377 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ce7eb588e44c3abcc6a0d1e12b784f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.296563 6769 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Bootstrap starting.
I20260811 07:06:54.296864 6770 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Bootstrap starting.
I20260811 07:06:54.297384 6769 tablet_bootstrap.cc:654] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.297551 6770 tablet_bootstrap.cc:654] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.297823 6769 log.cc:824] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:54.297750 6582 tablet_service.cc:1467] Processing CreateTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f (DEFAULT_TABLE table=test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.297925 6770 log.cc:824] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:54.298127 6582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ce7eb588e44c3abcc6a0d1e12b784f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.298651 6769 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: No bootstrap required, opened a new log
I20260811 07:06:54.298740 6769 ts_tablet_manager.cc:1397] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:54.298723 6770 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: No bootstrap required, opened a new log
I20260811 07:06:54.298854 6770 ts_tablet_manager.cc:1397] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:54.300872 6770 raft_consensus.cc:348] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.300868 6769 raft_consensus.cc:348] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.301043 6769 raft_consensus.cc:374] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.301043 6770 raft_consensus.cc:374] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.301084 6769 raft_consensus.cc:729] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160565cc916b47d2b015727a7096b568, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.301093 6770 raft_consensus.cc:729] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcd32232f6f4128b07ea490b61793d2, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.301210 6769 consensus_queue.cc:260] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.301225 6770 consensus_queue.cc:260] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.301510 6769 ts_tablet_manager.cc:1428] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:06:54.301510 6770 ts_tablet_manager.cc:1428] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:06:54.301570 6214 heartbeater.cc:503] Master 127.3.236.190:39805 was elected leader, sending a full tablet report...
I20260811 07:06:54.301652 6469 heartbeater.cc:503] Master 127.3.236.190:39805 was elected leader, sending a full tablet report...
I20260811 07:06:54.301995 6777 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Bootstrap starting.
I20260811 07:06:54.303009 6777 tablet_bootstrap.cc:654] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.303408 6777 log.cc:824] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:54.304116 6777 tablet_bootstrap.cc:492] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: No bootstrap required, opened a new log
I20260811 07:06:54.304203 6777 ts_tablet_manager.cc:1397] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:54.305760 6777 raft_consensus.cc:348] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.305943 6777 raft_consensus.cc:374] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.305997 6777 raft_consensus.cc:729] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083ee4009bcc4711a0b4907a97f5962f, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.306133 6777 consensus_queue.cc:260] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.306406 6777 ts_tablet_manager.cc:1428] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:54.306474 6728 heartbeater.cc:503] Master 127.3.236.190:39805 was elected leader, sending a full tablet report...
W20260811 07:06:54.341632 6470 tablet.cc:2369] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:54.399673 6788 raft_consensus.cc:482] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:54.399855 6788 raft_consensus.cc:504] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.400350 6788 leader_election.cc:288] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130:41575)
I20260811 07:06:54.404011 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.400955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:39975 (local address 127.3.236.130:41575)
0811 07:06:54.400997 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.400999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.401006 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:06:54.401032 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.401032 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.401040 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.401120 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.401267 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.401740 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403189 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.403630 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403720 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.403729 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.403903 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.403944 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:06:54.404021 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.400728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41575 (local address 127.3.236.131:39975)
0811 07:06:54.400876 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.400887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.400934 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.401126 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.401127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.401128 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.401206 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.401208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.401837 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.401838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.403103 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.403106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.403718 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.403804 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.403807 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.403894 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.403933 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":30,"thread_start_us":103,"threads_started":1}
I20260811 07:06:54.404084 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.400671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:55553 (local address 127.3.236.129:35699)
0811 07:06:54.400724 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.400726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.400738 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:06:54.400772 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.400773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.400782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.400877 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.401134 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.401946 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403308 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.403696 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403788 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.403798 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.403978 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.404016 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 07:06:54.404111 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.400608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:35699 (local address 127.3.236.131:55553)
0811 07:06:54.400654 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.400666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.400716 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.400945 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.400947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.400949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.401037 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.401039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.402052 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.402054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.403194 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.403197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.403787 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.403788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.403880 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.403884 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.403958 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.404003 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 07:06:54.404487 6383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ce7eb588e44c3abcc6a0d1e12b784f" candidate_uuid: "083ee4009bcc4711a0b4907a97f5962f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcd32232f6f4128b07ea490b61793d2" is_pre_election: true
I20260811 07:06:54.404645 6383 raft_consensus.cc:2454] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083ee4009bcc4711a0b4907a97f5962f in term 0.
I20260811 07:06:54.404835 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ce7eb588e44c3abcc6a0d1e12b784f" candidate_uuid: "083ee4009bcc4711a0b4907a97f5962f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568" is_pre_election: true
I20260811 07:06:54.404906 6527 leader_election.cc:302] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083ee4009bcc4711a0b4907a97f5962f, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.404980 6142 raft_consensus.cc:2454] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083ee4009bcc4711a0b4907a97f5962f in term 0.
I20260811 07:06:54.405025 6788 raft_consensus.cc:2781] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:54.405103 6788 raft_consensus.cc:482] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:54.405138 6788 raft_consensus.cc:3037] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.405860 6788 raft_consensus.cc:504] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.405992 6788 leader_election.cc:288] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [CANDIDATE]: Term 1 election: Requested vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130:41575)
I20260811 07:06:54.406149 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ce7eb588e44c3abcc6a0d1e12b784f" candidate_uuid: "083ee4009bcc4711a0b4907a97f5962f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568"
I20260811 07:06:54.406164 6383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ce7eb588e44c3abcc6a0d1e12b784f" candidate_uuid: "083ee4009bcc4711a0b4907a97f5962f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcd32232f6f4128b07ea490b61793d2"
I20260811 07:06:54.406235 6142 raft_consensus.cc:3037] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.406235 6383 raft_consensus.cc:3037] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.407296 6142 raft_consensus.cc:2454] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083ee4009bcc4711a0b4907a97f5962f in term 1.
I20260811 07:06:54.407296 6383 raft_consensus.cc:2454] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083ee4009bcc4711a0b4907a97f5962f in term 1.
I20260811 07:06:54.407505 6527 leader_election.cc:302] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083ee4009bcc4711a0b4907a97f5962f, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.407634 6788 raft_consensus.cc:2781] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:54.407807 6788 raft_consensus.cc:686] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 1 LEADER]: Becoming Leader. State: Replica: 083ee4009bcc4711a0b4907a97f5962f, State: Running, Role: LEADER
I20260811 07:06:54.407915 6788 consensus_queue.cc:237] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.408895 5860 catalog_manager.cc:5495] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f reported cstate change: term changed from 0 to 1, leader changed from <none> to 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131). New cstate: current_term: 1 leader_uuid: "083ee4009bcc4711a0b4907a97f5962f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } health_report { overall_health: HEALTHY } } }
W20260811 07:06:54.424139 6215 tablet.cc:2369] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:54.437016 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.434339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.3.236.131:35429)
0811 07:06:54.434382 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.434385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.434394 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:06:54.434487 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.434488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.434505 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.434570 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:54.434710 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.435181 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.435922 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.436110 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.436160 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.436395 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.436506 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.436602 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.436671 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.436694 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:06:54.439004 6383 raft_consensus.cc:1261] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Refusing update from remote peer 083ee4009bcc4711a0b4907a97f5962f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.439004 6142 raft_consensus.cc:1261] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Refusing update from remote peer 083ee4009bcc4711a0b4907a97f5962f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.439360 6788 consensus_queue.cc:1035] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [LEADER]: Connected to new peer: Peer: permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.439530 6790 consensus_queue.cc:1035] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.442072 6799 mvcc.cc:205] Tried to move back new op lower bound from 7317225531139104768 to 7317225531015581696. Current Snapshot: MvccSnapshot[applied={T|T < 7317225531139104768}]
I20260811 07:06:54.446295 5860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:54.446403 5860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:54.447229 5860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Sending DeleteTablet for 3 replicas of tablet e6ce7eb588e44c3abcc6a0d1e12b784f
I20260811 07:06:54.447562 6108 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:35424
I20260811 07:06:54.447616 6352 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:36284
I20260811 07:06:54.447770 6582 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ce7eb588e44c3abcc6a0d1e12b784f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:56460
I20260811 07:06:54.447901 6821 tablet_replica.cc:330] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: stopping tablet replica
I20260811 07:06:54.447974 6820 tablet_replica.cc:330] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: stopping tablet replica
I20260811 07:06:54.448103 6820 raft_consensus.cc:2229] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:54.448107 6821 raft_consensus.cc:2229] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:54.448068 5860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:06:54.448212 6820 raft_consensus.cc:2258] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 07:06:54.448283 5860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:54.448390 6822 tablet_replica.cc:330] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: stopping tablet replica
I20260811 07:06:54.448489 6822 raft_consensus.cc:2229] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:06:54.448608 6822 raft_consensus.cc:2258] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:54.448932 6821 raft_consensus.cc:2258] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:54.449077 6822 ts_tablet_manager.cc:1905] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.449525 6821 ts_tablet_manager.cc:1905] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.449790 6108 tablet_service.cc:1467] Processing CreateTablet for tablet c925fe0b841f4b1197b99985fdc81243 (DEFAULT_TABLE table=test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.449921 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c925fe0b841f4b1197b99985fdc81243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.450106 6820 ts_tablet_manager.cc:1905] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.451303 6769 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Bootstrap starting.
I20260811 07:06:54.451526 6821 ts_tablet_manager.cc:1918] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:06:54.451658 6821 log.cc:1196] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/e6ce7eb588e44c3abcc6a0d1e12b784f
I20260811 07:06:54.451788 6769 tablet_bootstrap.cc:654] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.452188 6821 ts_tablet_manager.cc:1939] T e6ce7eb588e44c3abcc6a0d1e12b784f P ffcd32232f6f4128b07ea490b61793d2: Deleting consensus metadata
I20260811 07:06:54.452378 6352 tablet_service.cc:1467] Processing CreateTablet for tablet c925fe0b841f4b1197b99985fdc81243 (DEFAULT_TABLE table=test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.452486 6352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c925fe0b841f4b1197b99985fdc81243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.453032 6822 ts_tablet_manager.cc:1918] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:06:54.453120 6822 log.cc:1196] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/e6ce7eb588e44c3abcc6a0d1e12b784f
I20260811 07:06:54.453382 6822 ts_tablet_manager.cc:1939] T e6ce7eb588e44c3abcc6a0d1e12b784f P 083ee4009bcc4711a0b4907a97f5962f: Deleting consensus metadata
I20260811 07:06:54.453595 6770 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Bootstrap starting.
I20260811 07:06:54.453779 6820 ts_tablet_manager.cc:1918] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:06:54.453845 6820 log.cc:1196] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/e6ce7eb588e44c3abcc6a0d1e12b784f
I20260811 07:06:54.453994 6582 tablet_service.cc:1467] Processing CreateTablet for tablet c925fe0b841f4b1197b99985fdc81243 (DEFAULT_TABLE table=test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:54.454105 6582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c925fe0b841f4b1197b99985fdc81243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.454147 6820 ts_tablet_manager.cc:1939] T e6ce7eb588e44c3abcc6a0d1e12b784f P 160565cc916b47d2b015727a7096b568: Deleting consensus metadata
I20260811 07:06:54.454185 6770 tablet_bootstrap.cc:654] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.455181 6777 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Bootstrap starting.
I20260811 07:06:54.455232 5832 catalog_manager.cc:4841] TS 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429): tablet e6ce7eb588e44c3abcc6a0d1e12b784f (table test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]) successfully deleted
I20260811 07:06:54.455443 5839 catalog_manager.cc:4841] TS 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699): tablet e6ce7eb588e44c3abcc6a0d1e12b784f (table test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]) successfully deleted
I20260811 07:06:54.455535 5839 catalog_manager.cc:4841] TS ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130:41575): tablet e6ce7eb588e44c3abcc6a0d1e12b784f (table test-table [id=631fac386ae542ddb7b5d4df8cc75ef2]) successfully deleted
I20260811 07:06:54.455636 6777 tablet_bootstrap.cc:654] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.456188 6769 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: No bootstrap required, opened a new log
I20260811 07:06:54.456245 6769 ts_tablet_manager.cc:1397] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 07:06:54.456321 6770 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: No bootstrap required, opened a new log
I20260811 07:06:54.456374 6770 ts_tablet_manager.cc:1397] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 07:06:54.456583 6770 raft_consensus.cc:348] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.456668 6770 raft_consensus.cc:374] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.456697 6770 raft_consensus.cc:729] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcd32232f6f4128b07ea490b61793d2, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.456696 6769 raft_consensus.cc:348] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.456740 6770 consensus_queue.cc:260] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.456785 6769 raft_consensus.cc:374] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.456815 6769 raft_consensus.cc:729] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160565cc916b47d2b015727a7096b568, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.456826 6770 ts_tablet_manager.cc:1428] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:54.456856 6769 consensus_queue.cc:260] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.456965 6769 ts_tablet_manager.cc:1428] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:54.457361 6777 tablet_bootstrap.cc:492] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: No bootstrap required, opened a new log
I20260811 07:06:54.457417 6777 ts_tablet_manager.cc:1397] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:06:54.457615 6777 raft_consensus.cc:348] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.457691 6777 raft_consensus.cc:374] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.457712 6777 raft_consensus.cc:729] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083ee4009bcc4711a0b4907a97f5962f, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.457751 6777 consensus_queue.cc:260] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.457834 6777 ts_tablet_manager.cc:1428] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:54.491392 6779 raft_consensus.cc:482] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:54.491501 6779 raft_consensus.cc:504] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.491931 6779 leader_election.cc:288] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429)
I20260811 07:06:54.495790 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.492152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35429 (local address 127.3.236.130:46853)
0811 07:06:54.492287 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.492300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.492347 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.492525 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.492527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.492529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.492610 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.492613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.493379 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.493381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.494720 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.494723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.495535 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.495536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.495624 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.495629 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.495680 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.495722 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 07:06:54.496013 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.492281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:52885 (local address 127.3.236.129:35699)
0811 07:06:54.492327 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.492329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.492563 (+ 234us) server_negotiation.cc:407] Connection header received
0811 07:06:54.492623 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.492624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.492634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.492683 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.492873 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.493644 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.495166 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.495590 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.495669 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.495680 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.495869 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.495939 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:06:54.496124 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.492324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:46853 (local address 127.3.236.131:35429)
0811 07:06:54.492372 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.492374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.492401 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:06:54.492432 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.492433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.492443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.492508 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:54.492689 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.493290 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.494822 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.495439 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.495518 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.495528 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.496001 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.496049 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:06:54.496382 6836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.492152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:35699 (local address 127.3.236.130:52885)
0811 07:06:54.492498 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.492508 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.492559 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.492727 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.492727 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.492729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:54.492800 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.492801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.493729 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.493730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.495077 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.495080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.495701 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.495702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.495782 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.495786 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.495845 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.496290 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":208,"thread_start_us":95,"threads_started":1}
I20260811 07:06:54.496347 6645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c925fe0b841f4b1197b99985fdc81243" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083ee4009bcc4711a0b4907a97f5962f" is_pre_election: true
I20260811 07:06:54.496503 6645 raft_consensus.cc:2454] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 0.
I20260811 07:06:54.496642 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c925fe0b841f4b1197b99985fdc81243" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568" is_pre_election: true
I20260811 07:06:54.496732 6142 raft_consensus.cc:2454] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 0.
I20260811 07:06:54.496708 6265 leader_election.cc:302] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083ee4009bcc4711a0b4907a97f5962f, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.496832 6779 raft_consensus.cc:2781] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:54.496874 6779 raft_consensus.cc:482] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:54.496912 6779 raft_consensus.cc:3037] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.497742 6779 raft_consensus.cc:504] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.497901 6779 leader_election.cc:288] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 election: Requested vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429)
W20260811 07:06:54.497938 6729 tablet.cc:2369] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:54.498150 6645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c925fe0b841f4b1197b99985fdc81243" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083ee4009bcc4711a0b4907a97f5962f"
I20260811 07:06:54.498252 6645 raft_consensus.cc:3037] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.498317 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c925fe0b841f4b1197b99985fdc81243" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568"
I20260811 07:06:54.498394 6142 raft_consensus.cc:3037] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.499256 6645 raft_consensus.cc:2454] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 1.
I20260811 07:06:54.499272 6142 raft_consensus.cc:2454] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 1.
I20260811 07:06:54.499444 6269 leader_election.cc:302] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160565cc916b47d2b015727a7096b568, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.499542 6779 raft_consensus.cc:2781] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:54.499756 6779 raft_consensus.cc:686] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 1 LEADER]: Becoming Leader. State: Replica: ffcd32232f6f4128b07ea490b61793d2, State: Running, Role: LEADER
I20260811 07:06:54.499835 6779 consensus_queue.cc:237] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.500576 5859 catalog_manager.cc:5495] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:54.545317 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.537234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.3.236.190:39805)
0811 07:06:54.537316 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.537318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.537457 (+ 139us) server_negotiation.cc:407] Connection header received
0811 07:06:54.537534 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.537535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.537545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.537607 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.538216 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.538698 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.539814 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.539990 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.540042 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.542498 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.542500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.542500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.542506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.543940 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.544422 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.544423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.544423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.544477 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.544808 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.544808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.544809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.544909 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.544957 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.545176 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.545250 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:06:54.545486 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.537167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39805 (local address 127.0.0.1:37240)
0811 07:06:54.537363 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.537378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.537461 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.537640 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.537642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.538060 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:54.538133 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.538136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.538784 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.538785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.539723 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.539726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.540066 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.540067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.540137 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.540743 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:54.540751 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:54.542403 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:54.544030 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.544037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.544039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.544353 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.544561 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:54.544562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:54.544562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:54.544725 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:54.544980 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:54.544988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:54.545050 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.545157 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.545350 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":36,"thread_start_us":113,"threads_started":1}
I20260811 07:06:54.548028 5859 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:37240:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:06:54.548308 5859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:54.549911 6582 tablet_service.cc:1467] Processing CreateTablet for tablet 54230f4cae9947eda60c3ba7ca46488f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d84a416dfec4c4b8a18fe91ce37ba7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:54.549949 6108 tablet_service.cc:1467] Processing CreateTablet for tablet 54230f4cae9947eda60c3ba7ca46488f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d84a416dfec4c4b8a18fe91ce37ba7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:54.550092 6582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54230f4cae9947eda60c3ba7ca46488f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.550092 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54230f4cae9947eda60c3ba7ca46488f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.551574 6352 tablet_service.cc:1467] Processing CreateTablet for tablet 54230f4cae9947eda60c3ba7ca46488f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d84a416dfec4c4b8a18fe91ce37ba7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:54.551703 6352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54230f4cae9947eda60c3ba7ca46488f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:54.552709 6769 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: Bootstrap starting.
I20260811 07:06:54.553170 6769 tablet_bootstrap.cc:654] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.553170 6770 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: Bootstrap starting.
I20260811 07:06:54.553706 6770 tablet_bootstrap.cc:654] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.553995 6769 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: No bootstrap required, opened a new log
I20260811 07:06:54.554064 6769 ts_tablet_manager.cc:1397] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:54.554224 6769 raft_consensus.cc:348] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.554303 6769 raft_consensus.cc:374] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.554333 6769 raft_consensus.cc:729] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160565cc916b47d2b015727a7096b568, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.554380 6769 consensus_queue.cc:260] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.554505 6769 ts_tablet_manager.cc:1428] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:54.554593 6778 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.554688 6778 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.554770 6770 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: No bootstrap required, opened a new log
I20260811 07:06:54.554803 6770 ts_tablet_manager.cc:1397] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:06:54.554966 6770 raft_consensus.cc:348] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.555003 6777 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: Bootstrap starting.
I20260811 07:06:54.555042 6770 raft_consensus.cc:374] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.555073 6770 raft_consensus.cc:729] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcd32232f6f4128b07ea490b61793d2, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.555117 6770 consensus_queue.cc:260] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.555214 6770 ts_tablet_manager.cc:1428] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:54.555327 6779 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.555384 6779 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.555552 6777 tablet_bootstrap.cc:654] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:54.556505 6777 tablet_bootstrap.cc:492] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: No bootstrap required, opened a new log
I20260811 07:06:54.556558 6777 ts_tablet_manager.cc:1397] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:06:54.556691 6777 raft_consensus.cc:348] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.556766 6777 raft_consensus.cc:374] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:54.556783 6777 raft_consensus.cc:729] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083ee4009bcc4711a0b4907a97f5962f, State: Initialized, Role: FOLLOWER
I20260811 07:06:54.556821 6777 consensus_queue.cc:260] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.556901 6777 ts_tablet_manager.cc:1428] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:54.557015 6814 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.557075 6814 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.698329 6779 raft_consensus.cc:482] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:54.698429 6779 raft_consensus.cc:504] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.698632 6779 leader_election.cc:288] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429)
I20260811 07:06:54.698925 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54230f4cae9947eda60c3ba7ca46488f" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568" is_pre_election: true
I20260811 07:06:54.698925 6645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54230f4cae9947eda60c3ba7ca46488f" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083ee4009bcc4711a0b4907a97f5962f" is_pre_election: true
I20260811 07:06:54.699043 6645 raft_consensus.cc:2454] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 0.
I20260811 07:06:54.699043 6142 raft_consensus.cc:2454] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 0.
I20260811 07:06:54.699230 6265 leader_election.cc:302] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083ee4009bcc4711a0b4907a97f5962f, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.699348 6779 raft_consensus.cc:2781] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:54.699397 6779 raft_consensus.cc:482] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:54.699412 6779 raft_consensus.cc:3037] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.700229 6779 raft_consensus.cc:504] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.700424 6779 leader_election.cc:288] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 election: Requested vote from peers 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699), 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429)
I20260811 07:06:54.700542 6142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54230f4cae9947eda60c3ba7ca46488f" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160565cc916b47d2b015727a7096b568"
I20260811 07:06:54.700625 6142 raft_consensus.cc:3037] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.700611 6645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54230f4cae9947eda60c3ba7ca46488f" candidate_uuid: "ffcd32232f6f4128b07ea490b61793d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083ee4009bcc4711a0b4907a97f5962f"
I20260811 07:06:54.700690 6645 raft_consensus.cc:3037] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:54.701438 6645 raft_consensus.cc:2454] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 1.
I20260811 07:06:54.701438 6142 raft_consensus.cc:2454] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffcd32232f6f4128b07ea490b61793d2 in term 1.
I20260811 07:06:54.701608 6269 leader_election.cc:302] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160565cc916b47d2b015727a7096b568, ffcd32232f6f4128b07ea490b61793d2; no voters:
I20260811 07:06:54.701751 6779 raft_consensus.cc:2781] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:54.701817 6779 raft_consensus.cc:686] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [term 1 LEADER]: Becoming Leader. State: Replica: ffcd32232f6f4128b07ea490b61793d2, State: Running, Role: LEADER
I20260811 07:06:54.701880 6779 consensus_queue.cc:237] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } }
I20260811 07:06:54.702184 6838 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: TxnStatusTablet state changed. Reason: New leader ffcd32232f6f4128b07ea490b61793d2. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.702286 6838 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:54.702526 6864 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:06:54.702661 6864 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:06:54.702627 5859 catalog_manager.cc:5495] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } health_report { overall_health: UNKNOWN } } }
I20260811 07:06:54.741122 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.738447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41575 (local address 127.0.0.1:36300)
0811 07:06:54.738527 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.738540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.738618 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.738831 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.738832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.738833 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:54.738887 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.738891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.739549 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.739551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.740127 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.740129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.740552 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.740553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.740630 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.740645 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.740937 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.740948 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.740989 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.741038 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:06:54.741110 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.738538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.3.236.130:41575)
0811 07:06:54.738618 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.738621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.738631 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:06:54.738703 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.738704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.738726 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.738821 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:54.738966 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.739454 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.740221 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.740472 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.740554 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.740749 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.740861 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.740942 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.741003 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.741026 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:06:54.743291 6142 raft_consensus.cc:1261] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Refusing update from remote peer ffcd32232f6f4128b07ea490b61793d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.743389 6645 raft_consensus.cc:1261] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Refusing update from remote peer ffcd32232f6f4128b07ea490b61793d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.743578 6779 consensus_queue.cc:1035] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.743708 6838 consensus_queue.cc:1035] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.744500 6778 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: TxnStatusTablet state changed. Reason: New leader ffcd32232f6f4128b07ea490b61793d2. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.744624 6778 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.744962 6814 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: TxnStatusTablet state changed. Reason: New leader ffcd32232f6f4128b07ea490b61793d2. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.745059 6814 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.745491 6801 mvcc.cc:205] Tried to move back new op lower bound from 7317225532386373632 to 7317225532219310080. Current Snapshot: MvccSnapshot[applied={T|T < 7317225532386373632}]
I20260811 07:06:54.745568 6779 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.745653 6779 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:54.745711 6800 mvcc.cc:205] Tried to move back new op lower bound from 7317225532386373632 to 7317225532219310080. Current Snapshot: MvccSnapshot[applied={T|T < 7317225532386373632}]
I20260811 07:06:54.745963 6778 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.746043 6778 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 160565cc916b47d2b015727a7096b568: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.747480 6838 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.747578 6838 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P ffcd32232f6f4128b07ea490b61793d2: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:54.748180 6814 tablet_replica.cc:439] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffcd32232f6f4128b07ea490b61793d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 } } peers { permanent_uuid: "ffcd32232f6f4128b07ea490b61793d2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41575 } } peers { permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 } } }
I20260811 07:06:54.748309 6814 tablet_replica.cc:442] T 54230f4cae9947eda60c3ba7ca46488f P 083ee4009bcc4711a0b4907a97f5962f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:54.750425 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.747795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.3.236.130:41575)
0811 07:06:54.747832 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.747834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.747840 (+ 6us) server_negotiation.cc:407] Connection header received
0811 07:06:54.747860 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.747861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.747873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.747925 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:54.748072 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.748837 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.749655 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.749963 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.750048 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.750183 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.750238 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.750299 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.750335 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.750355 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:06:54.755836 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.753130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47917 (local address 127.3.236.130:41575)
0811 07:06:54.753173 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.753175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.753331 (+ 156us) server_negotiation.cc:407] Connection header received
0811 07:06:54.753358 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.753359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.753368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.753436 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:54.753567 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.754289 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.755079 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.755370 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.755442 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.755603 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.755634 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.755687 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.755736 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.755754 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:06:54.756048 6874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.753069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41575 (local address 127.3.236.130:47917)
0811 07:06:54.753248 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:54.753260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:54.753323 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:54.753437 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:54.753437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:54.753438 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:54.753508 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.753509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.754371 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.754372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.755000 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:54.755002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.755470 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.755471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:54.755514 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.755527 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:54.755694 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:54.755696 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:06:54.755730 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 07:06:54.755763 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260811 07:06:54.758307 6142 raft_consensus.cc:1261] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Refusing update from remote peer ffcd32232f6f4128b07ea490b61793d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.758329 6645 raft_consensus.cc:1261] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Refusing update from remote peer ffcd32232f6f4128b07ea490b61793d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:54.758526 6838 consensus_queue.cc:1035] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "160565cc916b47d2b015727a7096b568" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.758630 6779 consensus_queue.cc:1035] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "083ee4009bcc4711a0b4907a97f5962f" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:54.773327 5860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:54.773434 5860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:54.774170 5860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91642af5af694d6f85197e23723903b5: Sending DeleteTablet for 3 replicas of tablet c925fe0b841f4b1197b99985fdc81243
I20260811 07:06:54.774435 6582 tablet_service.cc:1514] Processing DeleteTablet for tablet c925fe0b841f4b1197b99985fdc81243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:56460
I20260811 07:06:54.774439 6352 tablet_service.cc:1514] Processing DeleteTablet for tablet c925fe0b841f4b1197b99985fdc81243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:36284
I20260811 07:06:54.774547 6821 tablet_replica.cc:330] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: stopping tablet replica
I20260811 07:06:54.774557 6108 tablet_service.cc:1514] Processing DeleteTablet for tablet c925fe0b841f4b1197b99985fdc81243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:54 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:35424
I20260811 07:06:54.774602 6821 raft_consensus.cc:2229] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:06:54.774647 6822 tablet_replica.cc:330] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: stopping tablet replica
I20260811 07:06:54.774684 6821 raft_consensus.cc:2258] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:54.774703 6822 raft_consensus.cc:2229] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:54.774737 6822 raft_consensus.cc:2258] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:54.774806 6820 tablet_replica.cc:330] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: stopping tablet replica
I20260811 07:06:54.774865 6820 raft_consensus.cc:2229] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:54.774899 6820 raft_consensus.cc:2258] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:54.775050 6821 ts_tablet_manager.cc:1905] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.775254 6822 ts_tablet_manager.cc:1905] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.775849 6820 ts_tablet_manager.cc:1905] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:54.776798 6821 ts_tablet_manager.cc:1918] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:54.776830 6822 ts_tablet_manager.cc:1918] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:54.776854 6821 log.cc:1196] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/c925fe0b841f4b1197b99985fdc81243
I20260811 07:06:54.776877 6822 log.cc:1196] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/c925fe0b841f4b1197b99985fdc81243
I20260811 07:06:54.777083 6821 ts_tablet_manager.cc:1939] T c925fe0b841f4b1197b99985fdc81243 P ffcd32232f6f4128b07ea490b61793d2: Deleting consensus metadata
I20260811 07:06:54.777083 6822 ts_tablet_manager.cc:1939] T c925fe0b841f4b1197b99985fdc81243 P 083ee4009bcc4711a0b4907a97f5962f: Deleting consensus metadata
I20260811 07:06:54.777446 5839 catalog_manager.cc:4841] TS ffcd32232f6f4128b07ea490b61793d2 (127.3.236.130:41575): tablet c925fe0b841f4b1197b99985fdc81243 (table test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]) successfully deleted
I20260811 07:06:54.777577 6820 ts_tablet_manager.cc:1918] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:54.777630 6820 log.cc:1196] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/c925fe0b841f4b1197b99985fdc81243
I20260811 07:06:54.777848 6820 ts_tablet_manager.cc:1939] T c925fe0b841f4b1197b99985fdc81243 P 160565cc916b47d2b015727a7096b568: Deleting consensus metadata
I20260811 07:06:54.778072 5832 catalog_manager.cc:4841] TS 083ee4009bcc4711a0b4907a97f5962f (127.3.236.131:35429): tablet c925fe0b841f4b1197b99985fdc81243 (table test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]) successfully deleted
I20260811 07:06:54.778237 5839 catalog_manager.cc:4841] TS 160565cc916b47d2b015727a7096b568 (127.3.236.129:35699): tablet c925fe0b841f4b1197b99985fdc81243 (table test-table [id=d0abd4a4fda24f6f8561fcb77eb4259e]) successfully deleted
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[5788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:06:54.786731 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.778523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.3.236.129:35699)
0811 07:06:54.778564 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.778566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.778710 (+ 144us) server_negotiation.cc:407] Connection header received
0811 07:06:54.778787 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.778789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.778802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.778857 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.779420 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.779894 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.780743 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.780951 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.781037 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.783859 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.783867 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.783868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.783876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.785498 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.785859 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.785862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.785862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.785943 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.786161 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.786161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.786162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.786307 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.786460 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.786528 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.786619 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 07:06:54.787009 6206 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35426
I20260811 07:06:54.794981 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:54.787997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.3.236.190:39805)
0811 07:06:54.788076 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:54.788078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:54.788213 (+ 135us) server_negotiation.cc:407] Connection header received
0811 07:06:54.788320 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:54.788321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:54.788333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:54.788397 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:54.789054 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.789531 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.790511 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:54.790708 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:54.790766 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:54.792212 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:54.792214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:54.792214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:54.792221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:54.793835 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.794260 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.794261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.794261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.794317 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:54.794531 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:54.794531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:54.794532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:54.794637 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:54.794710 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:54.794855 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:06:54.794916 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:06:54.795303 5860 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37252
I20260811 07:06:54.796020 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5986
I20260811 07:06:54.808918 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6231
I20260811 07:06:54.822966 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6480
I20260811 07:06:54.833879 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5805
2026-08-11T07:06:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3531 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[6904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[6904](info): set up 2 sockets
Aug 11 07:06:54 dist-test-slave-2d19 krb5kdc[6904](info): commencing operation
krb5kdc: starting...
W20260811 07:06:56.177063 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:06:56.300830 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34025
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:34025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:56.386700 6921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:56.386958 6921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:56.387008 6921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:56.388227 6921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:06:56.388324 6921 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:06:56.388347 6921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:56.388370 6921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:56.388401 6921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:06:56.388427 6921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:06:56.389988 6921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:34025
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:56.390295 6921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:56.390538 6921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:56.395762 6926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.395823 6927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.395819 6930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:56.396011 6921 server_base.cc:1034] running on GCE node
I20260811 07:06:56.396337 6921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:56.396664 6921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:56.397850 6921 hybrid_clock.cc:648] HybridClock initialized: now 1786432016397835 us; error 47 us; skew 500 ppm
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:56.400679 6921 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:06:56.401176 6921 webserver.cc:460] Webserver started at http://127.3.236.190:42047/ using document root <none> and password file <none>
I20260811 07:06:56.401437 6921 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:56.401511 6921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:56.401667 6921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:56.402799 6921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "066a8bd627704ebf84324d1c6e719436"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "5d65f0950c9a79704f9e7d2658a67f8c"
server_key_iv: "612bb4e5926f15c9d8c8d07cf74f2156"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.403189 6921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "066a8bd627704ebf84324d1c6e719436"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "5d65f0950c9a79704f9e7d2658a67f8c"
server_key_iv: "612bb4e5926f15c9d8c8d07cf74f2156"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.404644 6921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:06:56.405395 6943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.405583 6921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:56.405697 6921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "066a8bd627704ebf84324d1c6e719436"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "5d65f0950c9a79704f9e7d2658a67f8c"
server_key_iv: "612bb4e5926f15c9d8c8d07cf74f2156"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.405776 6921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:56.419535 6921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:56.422304 6921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:56.422492 6921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:56.427423 6921 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:34025
I20260811 07:06:56.427501 7055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:34025 every 8 connection(s)
I20260811 07:06:56.427853 6921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:06:56.428627 7056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:56.430917 7056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Bootstrap starting.
I20260811 07:06:56.431474 7056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:56.431737 7056 log.cc:824] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:56.432535 7056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: No bootstrap required, opened a new log
I20260811 07:06:56.434602 7056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } }
I20260811 07:06:56.434744 7056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:56.434767 7056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066a8bd627704ebf84324d1c6e719436, State: Initialized, Role: FOLLOWER
I20260811 07:06:56.434909 7056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } }
I20260811 07:06:56.434995 7056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:06:56.435019 7056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:06:56.435046 7056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:56.435550 7056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } }
I20260811 07:06:56.435635 7056 leader_election.cc:302] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 066a8bd627704ebf84324d1c6e719436; no voters:
I20260811 07:06:56.435760 7056 leader_election.cc:288] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:06:56.435820 7059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:56.436029 7059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [term 1 LEADER]: Becoming Leader. State: Replica: 066a8bd627704ebf84324d1c6e719436, State: Running, Role: LEADER
I20260811 07:06:56.436049 7056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:06:56.436121 7059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } }
I20260811 07:06:56.436307 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6921
I20260811 07:06:56.436422 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:06:56.436618 4018 external_mini_cluster.cc:1428] Setting key 774fdabf26b0535a65b4570c728c55a6
I20260811 07:06:56.436575 7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 066a8bd627704ebf84324d1c6e719436. Latest consensus state: current_term: 1 leader_uuid: "066a8bd627704ebf84324d1c6e719436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } } }
I20260811 07:06:56.436554 7064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "066a8bd627704ebf84324d1c6e719436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066a8bd627704ebf84324d1c6e719436" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34025 } } }
I20260811 07:06:56.436677 7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:56.436682 7064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436 [sys.catalog]: This master's current role is: LEADER
I20260811 07:06:56.437703 7069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:06:56.438367 7069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:06:56.440104 7069 catalog_manager.cc:1347] Generated new cluster ID: e40a02994fd1497c8363211d11ee26ab
I20260811 07:06:56.440158 7069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:56.448292 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.437106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.3.236.190:34025)
0811 07:06:56.437283 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.437294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.437332 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:06:56.437459 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.437468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.437528 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.437684 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.438531 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.439808 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.440909 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.441520 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.441616 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.444397 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.444401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.444402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.444414 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.446346 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.446748 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.446751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.446751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.446864 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.447040 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.447041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.447041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.447311 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.447441 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.447908 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.448037 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260811 07:06:56.450712 7069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:06:56.451208 7069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:06:56.457443 7069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Generated new TSK 0
I20260811 07:06:56.457633 7069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:06:56.486914 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34025
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:56.570396 7103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:56.570649 7103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:56.570689 7103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:56.571835 7103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:56.571924 7103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:56.571964 7103 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:06:56.572013 7103 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:56.572049 7103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:06:56.573563 7103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:56.573843 7103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:56.574097 7103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:06:56.579428 7103 server_base.cc:1034] running on GCE node
W20260811 07:06:56.579362 7115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.579403 7120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.579361 7117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:56.579834 7103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:56.580096 7103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:56.581256 7103 hybrid_clock.cc:648] HybridClock initialized: now 1786432016581249 us; error 39 us; skew 500 ppm
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:56.583833 7103 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:06:56.584359 7103 webserver.cc:460] Webserver started at http://127.3.236.129:33045/ using document root <none> and password file <none>
I20260811 07:06:56.584566 7103 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:56.584610 7103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:56.584704 7103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:56.585690 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "e1277c98d84ed6c58d832219fbc4b536"
server_key_iv: "0697bb6ff3eb04dbbcfeb00224a6116f"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.585992 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "e1277c98d84ed6c58d832219fbc4b536"
server_key_iv: "0697bb6ff3eb04dbbcfeb00224a6116f"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.587265 7103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:06:56.587950 7131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.588167 7103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:56.588291 7103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "e1277c98d84ed6c58d832219fbc4b536"
server_key_iv: "0697bb6ff3eb04dbbcfeb00224a6116f"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.588364 7103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:56.609186 7103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:56.611222 7103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:56.611339 7103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:56.611847 7103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:56.611922 7103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.611949 7103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:56.611990 7103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.619449 7103 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:44799
I20260811 07:06:56.619565 7342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:44799 every 8 connection(s)
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:56.619887 7103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:06:56.622850 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7103
I20260811 07:06:56.623021 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:06:56.623260 4018 external_mini_cluster.cc:1428] Setting key cb0d56b2f264fcefa7a90833d1ee9f1c
I20260811 07:06:56.623279 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.612490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:41767 (local address 127.3.236.190:34025)
0811 07:06:56.612563 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.612566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.612788 (+ 222us) server_negotiation.cc:407] Connection header received
0811 07:06:56.613354 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.613355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.613370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.613433 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.614361 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.615174 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.616918 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.617276 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.617349 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.620345 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.620349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.620349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.620361 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.621899 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.622302 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.622303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.622304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.622354 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.622576 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.622576 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.622577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.622674 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.622727 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.623127 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.623215 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:06:56.623509 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.612464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.129:41767)
0811 07:06:56.612665 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.612709 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.613240 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.613504 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.613509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.613939 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.614271 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.614278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.615299 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.615301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.616795 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.616799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.617374 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.617375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.617548 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.618279 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.618290 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.620236 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.621996 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.622003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.622005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.622233 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.622414 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.622415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.622415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.622521 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.622758 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.622765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.622949 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.623089 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.623268 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":131,"threads_started":1}
I20260811 07:06:56.624104 7345 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34025
I20260811 07:06:56.624271 7345 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:56.624490 7345 heartbeater.cc:511] Master 127.3.236.190:34025 requested a full tablet report, sending...
I20260811 07:06:56.625029 6977 ts_manager.cc:194] Registered new tserver with Master: adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799)
I20260811 07:06:56.625901 6977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:41767
I20260811 07:06:56.631501 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.625201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:41095 (local address 127.3.236.190:34025)
0811 07:06:56.625242 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.625244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.625477 (+ 233us) server_negotiation.cc:407] Connection header received
0811 07:06:56.625551 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.625552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.625562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.625617 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.625937 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.626673 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.627525 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.627713 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.627769 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.628993 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.628994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.628994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.629001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.630369 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.630791 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.630792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.630792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.630843 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.631052 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.631053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.631053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.631155 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.631205 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.631381 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.631443 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:06:56.631606 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.625248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.129:41095)
0811 07:06:56.625399 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.625409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.625491 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.625623 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.625624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.625807 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.625858 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.625860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.626861 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.626862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.627435 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.627438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.627794 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.627795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.627924 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.628463 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.628469 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.628903 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.630465 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.630466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.630467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.630711 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.630908 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.630909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.630910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.630984 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.631225 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.631226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.631279 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.631357 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.631480 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:06:56.651806 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34025
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:56.736641 7367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:56.736909 7367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:56.736938 7367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:56.738297 7367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:56.738340 7367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:56.738361 7367 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:06:56.738405 7367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:56.738433 7367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:06:56.739961 7367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:56.740233 7367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:56.740521 7367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:56.745775 7375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:56.745824 7367 server_base.cc:1034] running on GCE node
W20260811 07:06:56.745769 7379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.745769 7376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:56.746258 7367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:56.746527 7367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:56.747694 7367 hybrid_clock.cc:648] HybridClock initialized: now 1786432016747675 us; error 35 us; skew 500 ppm
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:56.754858 7367 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:06:56.755451 7367 webserver.cc:460] Webserver started at http://127.3.236.130:35449/ using document root <none> and password file <none>
I20260811 07:06:56.755707 7367 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:56.755781 7367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:56.755932 7367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:56.757112 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "ba649352e56555e26b752b5413ba2f24"
server_key_iv: "5c6d16823827a1906ea464f1fa5e5273"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.757509 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "ba649352e56555e26b752b5413ba2f24"
server_key_iv: "5c6d16823827a1906ea464f1fa5e5273"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.758889 7367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:06:56.759639 7391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.759816 7367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:56.759930 7367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "ba649352e56555e26b752b5413ba2f24"
server_key_iv: "5c6d16823827a1906ea464f1fa5e5273"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.760017 7367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:56.773754 7367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:56.775748 7367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:56.775921 7367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:56.776635 7367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:56.776710 7367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.776742 7367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:56.776762 7367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:56.784759 7367 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:43163
I20260811 07:06:56.784826 7622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:43163 every 8 connection(s)
I20260811 07:06:56.785145 7367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:06:56.786950 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.776998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:49921 (local address 127.3.236.190:34025)
0811 07:06:56.777043 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.777045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.777253 (+ 208us) server_negotiation.cc:407] Connection header received
0811 07:06:56.777838 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.777839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.777853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.777905 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.778582 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.779114 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.780189 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.780457 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.780548 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.783958 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.783960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.783960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.783970 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.785766 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.786213 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.786214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.786214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.786270 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.786456 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.786457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.786457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.786570 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.786621 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.786822 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.786885 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:06:56.787168 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.776973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.130:49921)
0811 07:06:56.777128 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.777166 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.777715 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.777979 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.777983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.778242 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.778440 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.778444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.779221 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.779222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.780089 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.780091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.780567 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.780568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.780731 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.781282 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.781303 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.783843 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.785827 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.785830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.785831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.786138 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.786314 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.786314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.786315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.786406 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.786613 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.786615 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.786730 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.786809 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.786933 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260811 07:06:56.787616 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7367
I20260811 07:06:56.787716 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:06:56.787721 7623 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34025
I20260811 07:06:56.787812 7623 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:56.787941 4018 external_mini_cluster.cc:1428] Setting key 904eb978cf4f7fc8415f017e3990050e
I20260811 07:06:56.788030 7623 heartbeater.cc:511] Master 127.3.236.190:34025 requested a full tablet report, sending...
I20260811 07:06:56.788419 6974 ts_manager.cc:194] Registered new tserver with Master: 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:56.788890 6974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:49921
I20260811 07:06:56.795114 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.789077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:53803 (local address 127.3.236.190:34025)
0811 07:06:56.789118 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.789120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.789129 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:06:56.789221 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.789222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.789231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.789331 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.789712 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.790248 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.790987 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.791299 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.791378 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.792544 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.792545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.792546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.792552 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.793911 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.794328 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.794328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.794329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.794379 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.794597 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.794598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.794598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.794733 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.794786 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.794975 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.795050 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:06:56.795253 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.788882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.130:53803)
0811 07:06:56.789072 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.789084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.789137 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.789337 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.789338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.789547 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.789615 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.789617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.790345 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.790346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.790841 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.790842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.791381 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.791381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.791444 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.791949 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.791956 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.792444 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.794007 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.794008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.794009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.794253 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.794449 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.794451 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.794451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.794531 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.794807 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.794808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.794869 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.794962 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.795100 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:06:56.817216 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34025
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:06:56.902143 7640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:06:56.902398 7640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:06:56.902442 7640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:06:56.903596 7640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:06:56.903671 7640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:06:56.903705 7640 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:06:56.903735 7640 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:06:56.903769 7640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:06:56.905709 7640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:06:56.905998 7640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:06:56.906263 7640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:06:56.911652 7649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.911748 7650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:06:56.911774 7653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:06:56.912104 7640 server_base.cc:1034] running on GCE node
I20260811 07:06:56.912340 7640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:06:56.912603 7640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:06:56.913779 7640 hybrid_clock.cc:648] HybridClock initialized: now 1786432016913773 us; error 37 us; skew 500 ppm
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:06:56.916484 7640 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:06:56.916936 7640 webserver.cc:460] Webserver started at http://127.3.236.131:40471/ using document root <none> and password file <none>
I20260811 07:06:56.917145 7640 fs_manager.cc:359] Metadata directory not provided
I20260811 07:06:56.917186 7640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:06:56.917275 7640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:06:56.918295 7640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "9f2f0a6b9117448793b12c2d15173a45"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "76cf61b766d7306c9cb3806c3469ee87"
server_key_iv: "2c275c07b00cd4a6fb1fea6bb9b188c9"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.918602 7640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "9f2f0a6b9117448793b12c2d15173a45"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "76cf61b766d7306c9cb3806c3469ee87"
server_key_iv: "2c275c07b00cd4a6fb1fea6bb9b188c9"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.919914 7640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 07:06:56.920856 7665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.921027 7640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:06:56.921131 7640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "9f2f0a6b9117448793b12c2d15173a45"
format_stamp: "Formatted at 2026-08-11 07:06:56 on dist-test-slave-2d19"
server_key: "76cf61b766d7306c9cb3806c3469ee87"
server_key_iv: "2c275c07b00cd4a6fb1fea6bb9b188c9"
server_key_version: "encryptionkey@0"
I20260811 07:06:56.921190 7640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:06:56.934274 7640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:06:56.936489 7640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:06:56.936712 7640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:06:56.937374 7640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:06:56.937450 7640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.937505 7640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:06:56.937531 7640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:56.944851 7640 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:33873
I20260811 07:06:56.944921 7885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:33873 every 8 connection(s)
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:56.945314 7640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:06:56.948583 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.937978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:46781 (local address 127.3.236.190:34025)
0811 07:06:56.938050 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.938053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.938293 (+ 240us) server_negotiation.cc:407] Connection header received
0811 07:06:56.938898 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.938900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.938916 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.939010 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.939909 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.940791 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.942393 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.942755 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.942845 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.945759 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.945761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.945761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.945770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.946929 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.947396 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.947397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.947397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.947452 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.947700 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.947702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.947702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.947835 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.947911 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.948431 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.948516 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:06:56.948755 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.937972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.131:46781)
0811 07:06:56.938155 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.938197 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.938782 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.939077 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.939081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.939465 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.939795 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.939803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.940923 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.940924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.942288 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.942291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.942855 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.942857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.942974 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.943556 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.943566 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.945613 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.947059 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.947066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.947068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.947317 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.947497 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.947498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.947498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.947598 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.948044 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.948053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.948247 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.948407 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.948552 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260811 07:06:56.949329 7890 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34025
I20260811 07:06:56.949438 7890 heartbeater.cc:464] Registering TS with master...
I20260811 07:06:56.949642 7890 heartbeater.cc:511] Master 127.3.236.190:34025 requested a full tablet report, sending...
I20260811 07:06:56.950140 6974 ts_manager.cc:194] Registered new tserver with Master: 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873)
I20260811 07:06:56.950558 6974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:46781
I20260811 07:06:56.952898 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7640
I20260811 07:06:56.953003 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:06:56.953260 4018 external_mini_cluster.cc:1428] Setting key 5ce54b9d4cfd1a46b699aa461e43c4ad
I20260811 07:06:56.954255 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:06:56.956564 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.950620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:32843 (local address 127.3.236.190:34025)
0811 07:06:56.950668 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.950671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.950770 (+ 99us) server_negotiation.cc:407] Connection header received
0811 07:06:56.950830 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.950831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.950841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.950916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.951300 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.951772 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.952598 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.952779 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.952866 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.954049 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.954051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.954051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.954058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.955375 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.955732 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.955733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.955734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.955790 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.956017 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.956018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.956019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.956130 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.956202 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.956402 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.956473 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:06:56.956573 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.950556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.3.236.131:32843)
0811 07:06:56.950712 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.950724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.950780 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.950938 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.950939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.951148 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:56.951218 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.951220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.951901 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.951902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.952520 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.952523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.952868 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.952869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.952960 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.953531 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:56.953536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:56.953960 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:56.955475 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.955476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.955476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.955654 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.955872 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:56.955873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:56.955873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:56.955945 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:56.956217 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:56.956218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:56.956293 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.956362 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.956473 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:06:56 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:56.979609 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.972704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.3.236.190:34025)
0811 07:06:56.972768 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.972770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.972778 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:06:56.972807 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.972808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.972818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.972872 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:56.973462 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.974002 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.974773 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.974974 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.975036 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.977292 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:56.977294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:56.977294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:56.977301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:56.978531 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.978834 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.978835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.978835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.978887 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:56.979129 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:56.979130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:56.979131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:56.979260 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:56.979351 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.979437 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.979504 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:06:56.981324 6974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:56.982992 6974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:56.990330 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.3.236.131:33873)
0811 07:06:56.985973 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.985979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.985991 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:06:56.986044 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.986047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.986062 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.986155 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:56.986334 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.987522 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.989342 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.989861 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.989974 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990043 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.990169 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.990212 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
I20260811 07:06:56.990345 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:43163 (local address 127.0.0.1:55038)
0811 07:06:56.986264 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.986275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.986325 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.986895 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.986896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.986899 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:56.986941 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.986943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.987878 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.987879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.989034 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.989038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.990037 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.990038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.990104 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990108 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.990140 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.990230 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":35,"thread_start_us":96,"threads_started":1}
I20260811 07:06:56.990382 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:33873 (local address 127.0.0.1:33624)
0811 07:06:56.985912 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.985921 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.985959 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.986181 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.986184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.986186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:56.986259 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.986261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.987683 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.987684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.989258 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.989260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.989978 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.989979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.990059 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990074 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.990143 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.990223 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":84,"thread_start_us":91,"threads_started":1}
I20260811 07:06:56.990862 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.3.236.130:43163)
0811 07:06:56.986746 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.986749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.986757 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:06:56.986786 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.986787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.986798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.986883 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:56.986996 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.987770 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.989126 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.989926 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.990593 (+ 667us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990692 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.990718 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.990759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":78,"threads_started":1}
I20260811 07:06:56.991163 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.3.236.129:44799)
0811 07:06:56.986452 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:56.986457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:56.986467 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:06:56.986509 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:56.986511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:56.986525 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:56.986616 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:56.986806 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.987930 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.989225 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.989926 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.990016 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990080 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:56.990345 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 07:06:56.990854 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":102,"threads_started":1}
I20260811 07:06:56.991253 7757 tablet_service.cc:1467] Processing CreateTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c (DEFAULT_TABLE table=test-table [id=ef6f54254ef546e08cc64866cbad6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:56.991466 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:56.985519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.0.0.1:34148)
0811 07:06:56.985700 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:56.985741 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:56.985810 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:56.986613 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:56.986614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:56.986616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:56.986668 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.986669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.988011 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.988012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.989142 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:56.989146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:56.990114 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:56.990115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:56.990170 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:56.990173 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:06:56.990222 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:06:56.990256 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":61,"threads_started":1}
I20260811 07:06:56.991660 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750bcd6911b84b5db6b0f5436e1be88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:56.991575 7478 tablet_service.cc:1467] Processing CreateTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c (DEFAULT_TABLE table=test-table [id=ef6f54254ef546e08cc64866cbad6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:56.991932 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750bcd6911b84b5db6b0f5436e1be88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:56.995618 7941 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Bootstrap starting.
I20260811 07:06:56.996033 7942 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Bootstrap starting.
I20260811 07:06:56.996729 7941 tablet_bootstrap.cc:654] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:56.996588 7220 tablet_service.cc:1467] Processing CreateTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c (DEFAULT_TABLE table=test-table [id=ef6f54254ef546e08cc64866cbad6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:56.996796 7942 tablet_bootstrap.cc:654] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:56.996996 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750bcd6911b84b5db6b0f5436e1be88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:56.997188 7942 log.cc:824] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:56.997202 7941 log.cc:824] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:56.997991 7942 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: No bootstrap required, opened a new log
I20260811 07:06:56.998039 7941 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: No bootstrap required, opened a new log
I20260811 07:06:56.998091 7942 ts_tablet_manager.cc:1397] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:06:56.998137 7941 ts_tablet_manager.cc:1397] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:06:56.999711 7942 raft_consensus.cc:348] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:56.999878 7942 raft_consensus.cc:374] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:56.999922 7942 raft_consensus.cc:729] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87abc8e64a4bc9ad365f6e774bb557, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.000043 7942 consensus_queue.cc:260] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.000381 7942 ts_tablet_manager.cc:1428] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 07:06:57.000417 7623 heartbeater.cc:503] Master 127.3.236.190:34025 was elected leader, sending a full tablet report...
I20260811 07:06:57.001139 7941 raft_consensus.cc:348] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.001318 7941 raft_consensus.cc:374] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.001368 7941 raft_consensus.cc:729] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.001461 7950 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Bootstrap starting.
I20260811 07:06:57.001485 7941 consensus_queue.cc:260] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.001883 7941 ts_tablet_manager.cc:1428] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260811 07:06:57.002074 7890 heartbeater.cc:503] Master 127.3.236.190:34025 was elected leader, sending a full tablet report...
I20260811 07:06:57.002372 7950 tablet_bootstrap.cc:654] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.002861 7950 log.cc:824] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:06:57.003620 7950 tablet_bootstrap.cc:492] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: No bootstrap required, opened a new log
I20260811 07:06:57.003676 7950 ts_tablet_manager.cc:1397] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:57.005134 7950 raft_consensus.cc:348] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.005260 7950 raft_consensus.cc:374] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.005282 7950 raft_consensus.cc:729] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.005391 7950 consensus_queue.cc:260] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.005640 7950 ts_tablet_manager.cc:1428] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:06:57.005707 7345 heartbeater.cc:503] Master 127.3.236.190:34025 was elected leader, sending a full tablet report...
W20260811 07:06:57.035815 7624 tablet.cc:2369] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:57.045168 7952 raft_consensus.cc:482] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:57.045301 7952 raft_consensus.cc:504] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.045743 7952 leader_election.cc:288] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:57.049247 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.046085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38247 (local address 127.3.236.129:44799)
0811 07:06:57.046140 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.046143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.046158 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:06:57.046199 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.046200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.046208 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.046264 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:57.046478 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.047120 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048183 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.048814 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048897 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.048909 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.049113 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.049157 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:06:57.049233 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.046069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:50003 (local address 127.3.236.130:43163)
0811 07:06:57.046148 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.046151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.046533 (+ 382us) server_negotiation.cc:407] Connection header received
0811 07:06:57.046580 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.046580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.046588 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.046629 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:57.046779 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.047275 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048212 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.048862 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048929 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.048937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.049113 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.049156 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:06:57.049259 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.045993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.3.236.131:38247)
0811 07:06:57.046058 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.046073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.046131 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.046292 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.046293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.046296 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:57.046387 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.046390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.047207 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.047209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.048089 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.048092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.048915 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.049011 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.049020 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.049091 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.049145 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":12}
I20260811 07:06:57.049281 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.045995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:43163 (local address 127.3.236.131:50003)
0811 07:06:57.046405 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.046414 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.046525 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.046635 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.046635 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.046637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:57.046720 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.046721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.047362 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.047363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.048097 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.048100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.048934 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.048935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.049011 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.049020 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.049091 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.049145 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":114,"thread_start_us":223,"threads_started":1}
I20260811 07:06:57.049696 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750bcd6911b84b5db6b0f5436e1be88c" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" is_pre_election: true
I20260811 07:06:57.049687 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750bcd6911b84b5db6b0f5436e1be88c" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" is_pre_election: true
I20260811 07:06:57.049880 7515 raft_consensus.cc:2454] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:57.049880 7256 raft_consensus.cc:2454] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:57.050136 7666 leader_election.cc:302] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d87abc8e64a4bc9ad365f6e774bb557, 9f2f0a6b9117448793b12c2d15173a45; no voters:
I20260811 07:06:57.050304 7952 raft_consensus.cc:2781] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:57.050371 7952 raft_consensus.cc:482] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:57.050438 7952 raft_consensus.cc:3037] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.051185 7952 raft_consensus.cc:504] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.051337 7952 leader_election.cc:288] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Requested vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:57.051497 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750bcd6911b84b5db6b0f5436e1be88c" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
I20260811 07:06:57.051537 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750bcd6911b84b5db6b0f5436e1be88c" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
I20260811 07:06:57.051614 7515 raft_consensus.cc:3037] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.051609 7256 raft_consensus.cc:3037] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.052408 7256 raft_consensus.cc:2454] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:57.052657 7671 leader_election.cc:302] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:57.052789 7952 raft_consensus.cc:2781] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:57.052799 7515 raft_consensus.cc:2454] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:57.053042 7952 raft_consensus.cc:686] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Running, Role: LEADER
I20260811 07:06:57.053171 7952 consensus_queue.cc:237] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } }
I20260811 07:06:57.054203 6977 catalog_manager.cc:5495] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "9f2f0a6b9117448793b12c2d15173a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:06:57 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:57.088160 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.081061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.3.236.190:34025)
0811 07:06:57.081159 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.081161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.081284 (+ 123us) server_negotiation.cc:407] Connection header received
0811 07:06:57.081369 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.081369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.081380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.081460 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:57.081977 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.082470 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.083167 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.083348 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.083429 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.085575 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:57.085579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:57.085579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:57.085591 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:57.086825 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:57.087255 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:57.087256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:57.087257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:57.087316 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:57.087530 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:57.087530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:57.087531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:57.087642 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:57.087722 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.087914 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.088042 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":59}
I20260811 07:06:57.088231 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.080950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34025 (local address 127.0.0.1:39306)
0811 07:06:57.081197 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.081209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.081306 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.081466 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.081467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.081824 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:06:57.081870 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.081873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.082552 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.082553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.083080 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.083081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.083427 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.083428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.083482 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.083987 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:06:57.083992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:06:57.085473 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:06:57.086963 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:57.086972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:57.086974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:57.087174 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:57.087393 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:06:57.087394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:06:57.087395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:06:57.087455 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:06:57.087712 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:06:57.087721 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:06:57.087769 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.087888 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.088104 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":108,"thread_start_us":119,"threads_started":1}
I20260811 07:06:57.090950 6977 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:39306:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:06:57.091142 6977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:57.092706 7478 tablet_service.cc:1467] Processing CreateTablet for tablet ec8a56e3fce04792b1946526b49f8646 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b8a15dce13e4deb940e6b3ec5ce48e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:57.092784 7220 tablet_service.cc:1467] Processing CreateTablet for tablet ec8a56e3fce04792b1946526b49f8646 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b8a15dce13e4deb940e6b3ec5ce48e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:57.092903 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8a56e3fce04792b1946526b49f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.092934 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8a56e3fce04792b1946526b49f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.092960 7757 tablet_service.cc:1467] Processing CreateTablet for tablet ec8a56e3fce04792b1946526b49f8646 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b8a15dce13e4deb940e6b3ec5ce48e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:06:57.093154 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec8a56e3fce04792b1946526b49f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.094485 7950 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: Bootstrap starting.
I20260811 07:06:57.094519 7941 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: Bootstrap starting.
I20260811 07:06:57.094947 7941 tablet_bootstrap.cc:654] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.095206 7950 tablet_bootstrap.cc:654] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.095500 7942 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: Bootstrap starting.
I20260811 07:06:57.095638 7941 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: No bootstrap required, opened a new log
I20260811 07:06:57.095671 7941 ts_tablet_manager.cc:1397] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:06:57.095793 7941 raft_consensus.cc:348] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.095841 7941 raft_consensus.cc:374] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.095854 7941 raft_consensus.cc:729] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.095919 7941 consensus_queue.cc:260] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.096015 7941 ts_tablet_manager.cc:1428] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:57.096045 7942 tablet_bootstrap.cc:654] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.096118 7952 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.096225 7952 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.096519 7950 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: No bootstrap required, opened a new log
I20260811 07:06:57.096577 7950 ts_tablet_manager.cc:1397] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:06:57.096773 7950 raft_consensus.cc:348] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.096853 7950 raft_consensus.cc:374] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.096879 7950 raft_consensus.cc:729] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.096880 7942 tablet_bootstrap.cc:492] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: No bootstrap required, opened a new log
I20260811 07:06:57.096928 7942 ts_tablet_manager.cc:1397] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:06:57.096936 7950 consensus_queue.cc:260] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.097038 7950 ts_tablet_manager.cc:1428] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:57.097096 7942 raft_consensus.cc:348] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.097103 7954 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.097188 7942 raft_consensus.cc:374] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.097190 7954 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.097210 7942 raft_consensus.cc:729] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87abc8e64a4bc9ad365f6e774bb557, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.097257 7942 consensus_queue.cc:260] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.097345 7942 ts_tablet_manager.cc:1428] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:57.097429 7947 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.097523 7947 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 07:06:57.120677 7347 tablet.cc:2369] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:06:57.195924 7894 tablet.cc:2369] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:06:57.211503 7954 raft_consensus.cc:482] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:57.211606 7954 raft_consensus.cc:504] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.211973 7954 leader_election.cc:288] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163), 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873)
I20260811 07:06:57.214982 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.212180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:33873 (local address 127.3.236.129:33935)
0811 07:06:57.212342 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.212354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.212440 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.212613 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.212614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.212616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:57.212731 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.212733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.213363 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.213364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.214167 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.214169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.214709 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.214785 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.214790 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.214841 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.214873 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":46,"thread_start_us":91,"threads_started":1}
I20260811 07:06:57.214982 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.212288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:33935 (local address 127.3.236.131:33873)
0811 07:06:57.212335 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.212337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.212445 (+ 108us) server_negotiation.cc:407] Connection header received
0811 07:06:57.212504 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.212505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.212521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.212598 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:57.212804 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.213254 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214258 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.214623 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214679 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.214686 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.214868 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.214916 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:06:57.215060 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.212231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:52447 (local address 127.3.236.130:43163)
0811 07:06:57.212306 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.212309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.212423 (+ 114us) server_negotiation.cc:407] Connection header received
0811 07:06:57.212492 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.212493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.212503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.212561 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:06:57.212733 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.213247 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214308 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.214646 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214737 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.214745 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.214937 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.214976 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:06:57.215119 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.212156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:43163 (local address 127.3.236.129:52447)
0811 07:06:57.212330 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.212342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.212411 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.212585 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.212586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.212589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:06:57.212651 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.212655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.213338 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.213340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.214178 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.214181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.214762 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.214763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.214844 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.214849 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.214909 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.214950 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":54,"thread_start_us":85,"threads_started":1}
I20260811 07:06:57.215305 7802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8a56e3fce04792b1946526b49f8646" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2f0a6b9117448793b12c2d15173a45" is_pre_election: true
I20260811 07:06:57.215335 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8a56e3fce04792b1946526b49f8646" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" is_pre_election: true
I20260811 07:06:57.215408 7515 raft_consensus.cc:2454] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 0.
I20260811 07:06:57.215415 7802 raft_consensus.cc:2454] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 0.
I20260811 07:06:57.215600 7136 leader_election.cc:302] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:57.215785 7954 raft_consensus.cc:2781] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:57.215838 7954 raft_consensus.cc:482] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:57.215871 7954 raft_consensus.cc:3037] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.216621 7954 raft_consensus.cc:504] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.216773 7954 leader_election.cc:288] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163), 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873)
I20260811 07:06:57.216934 7802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8a56e3fce04792b1946526b49f8646" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2f0a6b9117448793b12c2d15173a45"
I20260811 07:06:57.216933 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec8a56e3fce04792b1946526b49f8646" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
I20260811 07:06:57.217007 7802 raft_consensus.cc:3037] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.217016 7515 raft_consensus.cc:3037] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.217811 7515 raft_consensus.cc:2454] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 1.
I20260811 07:06:57.217856 7802 raft_consensus.cc:2454] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 1.
I20260811 07:06:57.217993 7136 leader_election.cc:302] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:57.218107 7954 raft_consensus.cc:2781] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:57.218219 7954 raft_consensus.cc:686] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 LEADER]: Becoming Leader. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Running, Role: LEADER
I20260811 07:06:57.218322 7954 consensus_queue.cc:237] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.218760 7978 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: TxnStatusTablet state changed. Reason: New leader adfbe57f1eb2400b8eb4ef9890f22cd1. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.218855 7978 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:57.219066 7982 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:06:57.219189 7982 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:06:57.219158 6974 catalog_manager.cc:5495] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } health_report { overall_health: UNKNOWN } } }
I20260811 07:06:57.242780 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.240096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.3.236.129:44799)
0811 07:06:57.240152 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.240154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.240164 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:06:57.240247 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.240248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.240311 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.240398 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.240534 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.240968 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.242047 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.242222 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.242272 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.242482 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.242577 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.242634 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.242707 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.242724 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:06:57.242839 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.240013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.0.0.1:34154)
0811 07:06:57.240101 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.240115 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.240168 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.240415 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.240417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.240418 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:57.240471 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.240474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.241037 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.241038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.241952 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.241955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.242297 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.242298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.242371 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.242389 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.242640 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.242648 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.242705 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.242754 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 07:06:57.244658 7802 raft_consensus.cc:1261] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Refusing update from remote peer adfbe57f1eb2400b8eb4ef9890f22cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.244688 7515 raft_consensus.cc:1261] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Refusing update from remote peer adfbe57f1eb2400b8eb4ef9890f22cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.245018 7954 consensus_queue.cc:1035] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.245162 7978 consensus_queue.cc:1035] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.246402 7952 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: TxnStatusTablet state changed. Reason: New leader adfbe57f1eb2400b8eb4ef9890f22cd1. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.246450 7947 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: TxnStatusTablet state changed. Reason: New leader adfbe57f1eb2400b8eb4ef9890f22cd1. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.246517 7952 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.246541 7947 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.247623 7954 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.247715 7954 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:57.247786 7986 mvcc.cc:205] Tried to move back new op lower bound from 7317225542631186432 to 7317225542526713856. Current Snapshot: MvccSnapshot[applied={T|T < 7317225542631186432}]
I20260811 07:06:57.247960 7947 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.248061 7947 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 1d87abc8e64a4bc9ad365f6e774bb557: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.247649 7993 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.248296 7952 tablet_replica.cc:439] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } } }
I20260811 07:06:57.248370 7952 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P 9f2f0a6b9117448793b12c2d15173a45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:06:57.248441 7993 tablet_replica.cc:442] T ec8a56e3fce04792b1946526b49f8646 P adfbe57f1eb2400b8eb4ef9890f22cd1: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:06:57.249081 7988 mvcc.cc:205] Tried to move back new op lower bound from 7317225542631186432 to 7317225542526713856. Current Snapshot: MvccSnapshot[applied={T|T < 7317225542631186432 or (T in {7317225542631186432})}]
I20260811 07:06:57.253422 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.250956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.3.236.131:33873)
0811 07:06:57.251002 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.251004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.251014 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:06:57.251092 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.251092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.251110 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.251176 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.251300 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.251770 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.252634 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.252842 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.252929 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.253107 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.253211 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.253303 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.253326 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.253347 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:06:57.258424 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.256122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:57023 (local address 127.3.236.129:44799)
0811 07:06:57.256188 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.256190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.256214 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:06:57.256237 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.256238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.256247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.256331 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.256481 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.256946 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.257719 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.257922 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.257998 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.258179 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.258214 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.258277 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.258337 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.258357 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:06:57.258440 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.256028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.3.236.131:57023)
0811 07:06:57.256108 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.256120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.256166 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.256345 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.256346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.256347 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:57.256409 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.256410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.257058 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.257058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.257631 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.257633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.257997 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.257998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.258070 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.258084 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.258292 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.258294 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.258336 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.258376 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:06:57.260928 7256 raft_consensus.cc:1261] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.261155 7515 raft_consensus.cc:1261] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.261186 7952 consensus_queue.cc:1035] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.261368 7957 consensus_queue.cc:1035] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.270356 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.267555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:51889 (local address 127.3.236.131:33873)
0811 07:06:57.267672 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.267674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.267771 (+ 97us) server_negotiation.cc:407] Connection header received
0811 07:06:57.267845 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.267845 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.267854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.267893 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.268019 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.268531 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.269702 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.269874 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.269927 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.270147 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.270171 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.270210 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.270291 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.270308 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 07:06:57.270627 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.267516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:33873 (local address 127.3.236.129:51889)
0811 07:06:57.267711 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.267722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.267786 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.267906 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.267906 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.267908 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:57.267973 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.267974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.268602 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.268603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.269617 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.269620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.269953 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.269954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.270041 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.270060 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.270226 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.270228 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.270320 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.270365 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260811 07:06:57.278898 6977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:57.279018 6977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:57.279718 6977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Sending DeleteTablet for 3 replicas of tablet 750bcd6911b84b5db6b0f5436e1be88c
I20260811 07:06:57.280002 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:57 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34148
I20260811 07:06:57.280081 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:57 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:33624
I20260811 07:06:57.280328 7478 tablet_service.cc:1514] Processing DeleteTablet for tablet 750bcd6911b84b5db6b0f5436e1be88c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:57 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55038
I20260811 07:06:57.280371 8019 tablet_replica.cc:330] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: stopping tablet replica
I20260811 07:06:57.280573 8019 raft_consensus.cc:2229] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:57.280689 8019 raft_consensus.cc:2258] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:57.280821 6977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:57.280966 6977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:57.281355 8019 ts_tablet_manager.cc:1905] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:57.282087 7220 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 (DEFAULT_TABLE table=test-table [id=4d07fdfbeed3416e9d452226037a0b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:57.282209 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9b43b04b0e4cca810060adae001cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.283429 7950 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Bootstrap starting.
I20260811 07:06:57.283746 7757 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 (DEFAULT_TABLE table=test-table [id=4d07fdfbeed3416e9d452226037a0b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:57.283747 7478 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 (DEFAULT_TABLE table=test-table [id=4d07fdfbeed3416e9d452226037a0b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:57.283864 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9b43b04b0e4cca810060adae001cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.283865 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9b43b04b0e4cca810060adae001cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:57.283877 7950 tablet_bootstrap.cc:654] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.285245 7941 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Bootstrap starting.
I20260811 07:06:57.285245 7942 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Bootstrap starting.
I20260811 07:06:57.285724 7941 tablet_bootstrap.cc:654] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.285724 7942 tablet_bootstrap.cc:654] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:57.286613 8020 tablet_replica.cc:330] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: stopping tablet replica
I20260811 07:06:57.286752 8020 raft_consensus.cc:2229] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:06:57.286873 8020 raft_consensus.cc:2258] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:57.286932 7941 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: No bootstrap required, opened a new log
I20260811 07:06:57.286968 7941 ts_tablet_manager.cc:1397] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260811 07:06:57.287122 8021 tablet_replica.cc:330] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: stopping tablet replica
I20260811 07:06:57.287170 7941 raft_consensus.cc:348] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.287269 7941 raft_consensus.cc:374] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.287227 8021 raft_consensus.cc:2229] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:57.287293 7941 raft_consensus.cc:729] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.287338 8021 raft_consensus.cc:2258] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:57.287351 8020 ts_tablet_manager.cc:1905] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:57.287348 7941 consensus_queue.cc:260] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.287456 7941 ts_tablet_manager.cc:1428] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:57.287472 8019 ts_tablet_manager.cc:1918] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:57.287572 8019 log.cc:1196] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/750bcd6911b84b5db6b0f5436e1be88c
I20260811 07:06:57.287887 8021 ts_tablet_manager.cc:1905] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:57.288280 8019 ts_tablet_manager.cc:1939] T 750bcd6911b84b5db6b0f5436e1be88c P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting consensus metadata
I20260811 07:06:57.288767 6952 catalog_manager.cc:4841] TS adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799): tablet 750bcd6911b84b5db6b0f5436e1be88c (table test-table [id=ef6f54254ef546e08cc64866cbad6efe]) successfully deleted
I20260811 07:06:57.288825 7942 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: No bootstrap required, opened a new log
I20260811 07:06:57.288870 7942 ts_tablet_manager.cc:1397] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 07:06:57.289027 7942 raft_consensus.cc:348] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.289095 7942 raft_consensus.cc:374] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.289116 7942 raft_consensus.cc:729] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87abc8e64a4bc9ad365f6e774bb557, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.289165 7942 consensus_queue.cc:260] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.289254 7942 ts_tablet_manager.cc:1428] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:57.289397 8020 ts_tablet_manager.cc:1918] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:57.289467 8020 log.cc:1196] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/750bcd6911b84b5db6b0f5436e1be88c
I20260811 07:06:57.289744 8021 ts_tablet_manager.cc:1918] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:57.289806 8021 log.cc:1196] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/750bcd6911b84b5db6b0f5436e1be88c
I20260811 07:06:57.290038 8020 ts_tablet_manager.cc:1939] T 750bcd6911b84b5db6b0f5436e1be88c P 9f2f0a6b9117448793b12c2d15173a45: Deleting consensus metadata
I20260811 07:06:57.290081 8021 ts_tablet_manager.cc:1939] T 750bcd6911b84b5db6b0f5436e1be88c P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting consensus metadata
I20260811 07:06:57.290465 6945 catalog_manager.cc:4841] TS 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163): tablet 750bcd6911b84b5db6b0f5436e1be88c (table test-table [id=ef6f54254ef546e08cc64866cbad6efe]) successfully deleted
I20260811 07:06:57.290465 6951 catalog_manager.cc:4841] TS 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873): tablet 750bcd6911b84b5db6b0f5436e1be88c (table test-table [id=ef6f54254ef546e08cc64866cbad6efe]) successfully deleted
I20260811 07:06:57.292620 7950 tablet_bootstrap.cc:492] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: No bootstrap required, opened a new log
I20260811 07:06:57.292690 7950 ts_tablet_manager.cc:1397] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260811 07:06:57.292824 7950 raft_consensus.cc:348] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.292901 7950 raft_consensus.cc:374] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:57.292929 7950 raft_consensus.cc:729] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Initialized, Role: FOLLOWER
I20260811 07:06:57.292974 7950 consensus_queue.cc:260] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.293076 7950 ts_tablet_manager.cc:1428] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:57.379822 7993 raft_consensus.cc:482] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:57.379927 7993 raft_consensus.cc:504] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.380100 7993 leader_election.cc:288] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163), 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873)
I20260811 07:06:57.380340 7802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9b43b04b0e4cca810060adae001cf5" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2f0a6b9117448793b12c2d15173a45" is_pre_election: true
I20260811 07:06:57.380333 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9b43b04b0e4cca810060adae001cf5" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" is_pre_election: true
I20260811 07:06:57.380426 7802 raft_consensus.cc:2454] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 0.
I20260811 07:06:57.380431 7515 raft_consensus.cc:2454] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 0.
I20260811 07:06:57.380589 7136 leader_election.cc:302] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:57.380681 7993 raft_consensus.cc:2781] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:57.380710 7993 raft_consensus.cc:482] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:57.380728 7993 raft_consensus.cc:3037] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.381827 7993 raft_consensus.cc:504] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.381974 7993 leader_election.cc:288] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163), 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873)
I20260811 07:06:57.382162 7802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9b43b04b0e4cca810060adae001cf5" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2f0a6b9117448793b12c2d15173a45"
I20260811 07:06:57.382162 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9b43b04b0e4cca810060adae001cf5" candidate_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
I20260811 07:06:57.382228 7802 raft_consensus.cc:3037] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.382228 7515 raft_consensus.cc:3037] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:57.382948 7802 raft_consensus.cc:2454] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 1.
I20260811 07:06:57.382985 7515 raft_consensus.cc:2454] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adfbe57f1eb2400b8eb4ef9890f22cd1 in term 1.
I20260811 07:06:57.383144 7132 leader_election.cc:302] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d87abc8e64a4bc9ad365f6e774bb557, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:57.383276 7993 raft_consensus.cc:2781] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:57.383335 7993 raft_consensus.cc:686] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 LEADER]: Becoming Leader. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Running, Role: LEADER
I20260811 07:06:57.383391 7993 consensus_queue.cc:237] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:57.383980 6977 catalog_manager.cc:5495] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } health_report { overall_health: UNKNOWN } } }
I20260811 07:06:57.407938 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.405655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.3.236.129:44799)
0811 07:06:57.405695 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.405698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.405732 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:06:57.405809 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.405810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.405822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.405877 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.405984 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.406455 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.407197 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.407407 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.407483 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.407628 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.407664 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.407736 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.407803 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.407826 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:06:57.412396 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.410267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:35195 (local address 127.3.236.129:44799)
0811 07:06:57.410313 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:57.410314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:57.410321 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:06:57.410339 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:57.410339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:57.410347 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:57.410418 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:57.410524 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.410968 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.411668 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.411857 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.411933 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.412073 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.412100 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.412178 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:57.412240 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 07:06:57.412312 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":29}
I20260811 07:06:57.412405 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:57.410122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.3.236.129:35195)
0811 07:06:57.410171 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:06:57.410181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:06:57.410221 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:06:57.410421 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:06:57.410422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:06:57.410423 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:06:57.410458 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.410460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.411050 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.411051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.411576 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:06:57.411578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:06:57.411930 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:06:57.411931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:06:57.411986 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:57.411992 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:06:57.412182 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:06:57.412183 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:06:57.412236 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 07:06:57.412333 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":44}
I20260811 07:06:57.414603 7802 raft_consensus.cc:1261] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Refusing update from remote peer adfbe57f1eb2400b8eb4ef9890f22cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.414717 7515 raft_consensus.cc:1261] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Refusing update from remote peer adfbe57f1eb2400b8eb4ef9890f22cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:57.414834 7993 consensus_queue.cc:1035] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.414933 7978 consensus_queue.cc:1035] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:57.416700 7987 mvcc.cc:205] Tried to move back new op lower bound from 7317225543328759808 to 7317225543202693120. Current Snapshot: MvccSnapshot[applied={T|T < 7317225543328759808}]
W20260811 07:06:57.420572 7986 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 07:06:58.941541 6974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:58.941663 6974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:58.942518 6974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Sending DeleteTablet for 3 replicas of tablet 3c9b43b04b0e4cca810060adae001cf5
I20260811 07:06:58.942862 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:58 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34148
I20260811 07:06:58.942862 7478 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:58 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55038
I20260811 07:06:58.943070 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9b43b04b0e4cca810060adae001cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:58 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:33624
I20260811 07:06:58.943306 8078 tablet_replica.cc:330] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: stopping tablet replica
I20260811 07:06:58.943382 8078 raft_consensus.cc:2229] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:58.943425 8078 raft_consensus.cc:2258] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:58.943717 8078 ts_tablet_manager.cc:1905] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:58.945907 8078 ts_tablet_manager.cc:1918] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:06:58.945969 8078 log.cc:1196] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/3c9b43b04b0e4cca810060adae001cf5
I20260811 07:06:58.946215 8078 ts_tablet_manager.cc:1939] T 3c9b43b04b0e4cca810060adae001cf5 P 9f2f0a6b9117448793b12c2d15173a45: Deleting consensus metadata
I20260811 07:06:58.946524 6951 catalog_manager.cc:4841] TS 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873): tablet 3c9b43b04b0e4cca810060adae001cf5 (table test-table [id=4d07fdfbeed3416e9d452226037a0b49]) successfully deleted
I20260811 07:06:58.946712 8076 tablet_replica.cc:330] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: stopping tablet replica
I20260811 07:06:58.946797 8076 raft_consensus.cc:2229] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:58.946863 8076 raft_consensus.cc:2258] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:58.946983 8077 tablet_replica.cc:330] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: stopping tablet replica
I20260811 07:06:58.947060 8077 raft_consensus.cc:2229] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:06:58.947167 8077 raft_consensus.cc:2258] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:58.947158 8076 ts_tablet_manager.cc:1905] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:58.947409 8077 ts_tablet_manager.cc:1905] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:58.948886 8077 ts_tablet_manager.cc:1918] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:06:58.948962 8077 log.cc:1196] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/3c9b43b04b0e4cca810060adae001cf5
I20260811 07:06:58.949230 8077 ts_tablet_manager.cc:1939] T 3c9b43b04b0e4cca810060adae001cf5 P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting consensus metadata
I20260811 07:06:58.949223 8076 ts_tablet_manager.cc:1918] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:06:58.949311 8076 log.cc:1196] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/3c9b43b04b0e4cca810060adae001cf5
I20260811 07:06:58.949570 6952 catalog_manager.cc:4841] TS adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799): tablet 3c9b43b04b0e4cca810060adae001cf5 (table test-table [id=4d07fdfbeed3416e9d452226037a0b49]) successfully deleted
I20260811 07:06:58.949609 8076 ts_tablet_manager.cc:1939] T 3c9b43b04b0e4cca810060adae001cf5 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting consensus metadata
I20260811 07:06:58.950078 6945 catalog_manager.cc:4841] TS 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163): tablet 3c9b43b04b0e4cca810060adae001cf5 (table test-table [id=4d07fdfbeed3416e9d452226037a0b49]) successfully deleted
Aug 11 07:06:58 dist-test-slave-2d19 krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:06:58 dist-test-slave-2d19 krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:06:58.969777 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:58.962395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.3.236.190:34025)
0811 07:06:58.962573 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:58.962575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:58.962604 (+ 29us) server_negotiation.cc:407] Connection header received
0811 07:06:58.962630 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:58.962631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:58.962642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:58.962695 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:06:58.963315 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:58.963860 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:58.964688 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:58.964966 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:58.965021 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:58.967282 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:06:58.967284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:06:58.967284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:06:58.967291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:06:58.968846 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:58.969143 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:58.969144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:58.969145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:58.969200 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:06:58.969375 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:06:58.969375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:06:58.969376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:06:58.969478 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:06:58.969556 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:58.969650 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 07:06:58.969710 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260811 07:06:58.971293 6974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:58.971469 6974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:58.972976 7220 tablet_service.cc:1467] Processing CreateTablet for tablet c195997ca5f74edf9939ca522bf1bfed (DEFAULT_TABLE table=test-table [id=7193dc41312b4204a23099ccbe6b1b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:58.972992 7478 tablet_service.cc:1467] Processing CreateTablet for tablet c195997ca5f74edf9939ca522bf1bfed (DEFAULT_TABLE table=test-table [id=7193dc41312b4204a23099ccbe6b1b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:58.973030 7757 tablet_service.cc:1467] Processing CreateTablet for tablet c195997ca5f74edf9939ca522bf1bfed (DEFAULT_TABLE table=test-table [id=7193dc41312b4204a23099ccbe6b1b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:58.973141 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c195997ca5f74edf9939ca522bf1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:58.973145 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c195997ca5f74edf9939ca522bf1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:58.973148 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c195997ca5f74edf9939ca522bf1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:58.974502 8098 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Bootstrap starting.
I20260811 07:06:58.974552 8097 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Bootstrap starting.
I20260811 07:06:58.974685 8100 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Bootstrap starting.
I20260811 07:06:58.974947 8097 tablet_bootstrap.cc:654] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:58.974948 8098 tablet_bootstrap.cc:654] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:58.975133 8100 tablet_bootstrap.cc:654] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:58.975630 8098 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: No bootstrap required, opened a new log
I20260811 07:06:58.975692 8098 ts_tablet_manager.cc:1397] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:58.975771 8097 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: No bootstrap required, opened a new log
I20260811 07:06:58.975831 8097 ts_tablet_manager.cc:1397] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:58.975848 8098 raft_consensus.cc:348] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.975934 8098 raft_consensus.cc:374] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:58.975948 8100 tablet_bootstrap.cc:492] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: No bootstrap required, opened a new log
I20260811 07:06:58.975960 8098 raft_consensus.cc:729] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87abc8e64a4bc9ad365f6e774bb557, State: Initialized, Role: FOLLOWER
I20260811 07:06:58.975989 8100 ts_tablet_manager.cc:1397] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:58.975948 8097 raft_consensus.cc:348] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.976009 8098 consensus_queue.cc:260] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.976058 8097 raft_consensus.cc:374] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:58.976081 8097 raft_consensus.cc:729] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Initialized, Role: FOLLOWER
I20260811 07:06:58.976104 8100 raft_consensus.cc:348] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.976173 8100 raft_consensus.cc:374] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:58.976154 8097 consensus_queue.cc:260] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.976195 8100 raft_consensus.cc:729] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Initialized, Role: FOLLOWER
I20260811 07:06:58.976198 8098 ts_tablet_manager.cc:1428] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:58.976290 8097 ts_tablet_manager.cc:1428] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:06:58.976639 8100 consensus_queue.cc:260] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:58.976991 8100 ts_tablet_manager.cc:1428] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:59.239522 8110 raft_consensus.cc:482] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:59.239637 8110 raft_consensus.cc:504] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.239836 8110 leader_election.cc:288] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:59.240114 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c195997ca5f74edf9939ca522bf1bfed" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" is_pre_election: true
I20260811 07:06:59.240109 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c195997ca5f74edf9939ca522bf1bfed" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" is_pre_election: true
I20260811 07:06:59.240216 7256 raft_consensus.cc:2454] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:59.240216 7515 raft_consensus.cc:2454] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:59.240427 7671 leader_election.cc:302] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:59.240540 8110 raft_consensus.cc:2781] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:59.240585 8110 raft_consensus.cc:482] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:59.240618 8110 raft_consensus.cc:3037] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.241323 8110 raft_consensus.cc:504] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.241472 8110 leader_election.cc:288] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Requested vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:59.241616 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c195997ca5f74edf9939ca522bf1bfed" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
I20260811 07:06:59.241657 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c195997ca5f74edf9939ca522bf1bfed" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
I20260811 07:06:59.241714 7256 raft_consensus.cc:3037] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.241729 7515 raft_consensus.cc:3037] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.242523 7256 raft_consensus.cc:2454] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:59.242523 7515 raft_consensus.cc:2454] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:59.242717 7666 leader_election.cc:302] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d87abc8e64a4bc9ad365f6e774bb557, 9f2f0a6b9117448793b12c2d15173a45; no voters:
I20260811 07:06:59.242839 8110 raft_consensus.cc:2781] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:59.242990 8110 raft_consensus.cc:686] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Running, Role: LEADER
I20260811 07:06:59.243037 8110 consensus_queue.cc:237] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.243825 6972 catalog_manager.cc:5495] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "9f2f0a6b9117448793b12c2d15173a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } health_report { overall_health: HEALTHY } } }
I20260811 07:06:59.263043 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:06:59.260213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.3.236.131:33873)
0811 07:06:59.260402 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:06:59.260404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:06:59.260411 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:06:59.260432 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:06:59.260433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:06:59.260451 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:06:59.260583 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:06:59.260760 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:59.261391 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:59.262281 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:06:59.262532 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:06:59.262641 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:06:59.262784 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:06:59.262820 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:06:59.262872 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:06:59.262927 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 07:06:59.262950 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":133,"threads_started":1}
I20260811 07:06:59.265591 7515 raft_consensus.cc:1261] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:59.265753 7256 raft_consensus.cc:1261] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:59.265836 8113 consensus_queue.cc:1035] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:59.265949 8110 consensus_queue.cc:1035] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:59.276013 6972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:59.276116 6972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:06:59.276755 6972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Sending DeleteTablet for 3 replicas of tablet c195997ca5f74edf9939ca522bf1bfed
I20260811 07:06:59.277007 7478 tablet_service.cc:1514] Processing DeleteTablet for tablet c195997ca5f74edf9939ca522bf1bfed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55038
I20260811 07:06:59.277041 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet c195997ca5f74edf9939ca522bf1bfed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34148
I20260811 07:06:59.277088 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet c195997ca5f74edf9939ca522bf1bfed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:06:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:33624
I20260811 07:06:59.277119 8076 tablet_replica.cc:330] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: stopping tablet replica
I20260811 07:06:59.277154 8077 tablet_replica.cc:330] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: stopping tablet replica
I20260811 07:06:59.277168 8078 tablet_replica.cc:330] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: stopping tablet replica
I20260811 07:06:59.277166 8076 raft_consensus.cc:2229] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:59.277206 8076 raft_consensus.cc:2258] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:59.277208 8077 raft_consensus.cc:2229] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:06:59.277216 8078 raft_consensus.cc:2229] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:06:59.277249 8077 raft_consensus.cc:2258] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:59.277275 8078 raft_consensus.cc:2258] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:06:59.277552 8076 ts_tablet_manager.cc:1905] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:59.277654 8077 ts_tablet_manager.cc:1905] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:59.277678 6972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:06:59.277820 6972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:06:59.279122 7220 tablet_service.cc:1467] Processing CreateTablet for tablet 965efe0c30a840b79b1215797794d326 (DEFAULT_TABLE table=test-table [id=19951b7344474d4f89e2201c40eba523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:59.279207 8078 ts_tablet_manager.cc:1905] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:06:59.279242 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965efe0c30a840b79b1215797794d326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:59.279390 8076 ts_tablet_manager.cc:1918] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:59.279405 8077 ts_tablet_manager.cc:1918] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:59.279454 8077 log.cc:1196] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/c195997ca5f74edf9939ca522bf1bfed
I20260811 07:06:59.279462 8076 log.cc:1196] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/c195997ca5f74edf9939ca522bf1bfed
I20260811 07:06:59.279695 8077 ts_tablet_manager.cc:1939] T c195997ca5f74edf9939ca522bf1bfed P adfbe57f1eb2400b8eb4ef9890f22cd1: Deleting consensus metadata
I20260811 07:06:59.279701 8076 ts_tablet_manager.cc:1939] T c195997ca5f74edf9939ca522bf1bfed P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting consensus metadata
I20260811 07:06:59.280100 7757 tablet_service.cc:1467] Processing CreateTablet for tablet 965efe0c30a840b79b1215797794d326 (DEFAULT_TABLE table=test-table [id=19951b7344474d4f89e2201c40eba523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:59.280134 6945 catalog_manager.cc:4841] TS 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163): tablet c195997ca5f74edf9939ca522bf1bfed (table test-table [id=7193dc41312b4204a23099ccbe6b1b78]) successfully deleted
I20260811 07:06:59.280212 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965efe0c30a840b79b1215797794d326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:59.280347 7478 tablet_service.cc:1467] Processing CreateTablet for tablet 965efe0c30a840b79b1215797794d326 (DEFAULT_TABLE table=test-table [id=19951b7344474d4f89e2201c40eba523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:06:59.280475 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965efe0c30a840b79b1215797794d326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:06:59.280666 8097 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1: Bootstrap starting.
I20260811 07:06:59.280987 8097 tablet_bootstrap.cc:654] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:59.281028 6952 catalog_manager.cc:4841] TS adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799): tablet c195997ca5f74edf9939ca522bf1bfed (table test-table [id=7193dc41312b4204a23099ccbe6b1b78]) successfully deleted
I20260811 07:06:59.281543 8097 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1: No bootstrap required, opened a new log
I20260811 07:06:59.281567 8100 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Bootstrap starting.
I20260811 07:06:59.281602 8097 ts_tablet_manager.cc:1397] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:59.281747 8098 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Bootstrap starting.
I20260811 07:06:59.281725 8097 raft_consensus.cc:348] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.281800 8097 raft_consensus.cc:374] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:59.281826 8097 raft_consensus.cc:729] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfbe57f1eb2400b8eb4ef9890f22cd1, State: Initialized, Role: FOLLOWER
I20260811 07:06:59.281868 8097 consensus_queue.cc:260] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.281935 8100 tablet_bootstrap.cc:654] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:59.281975 8097 ts_tablet_manager.cc:1428] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:59.282439 8098 tablet_bootstrap.cc:654] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Neither blocks nor log segments found. Creating new log.
I20260811 07:06:59.282577 8100 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: No bootstrap required, opened a new log
I20260811 07:06:59.282591 8078 ts_tablet_manager.cc:1918] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:06:59.282685 8100 ts_tablet_manager.cc:1397] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:06:59.282688 8078 log.cc:1196] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/c195997ca5f74edf9939ca522bf1bfed
I20260811 07:06:59.282811 8100 raft_consensus.cc:348] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.282888 8100 raft_consensus.cc:374] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:59.282914 8100 raft_consensus.cc:729] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Initialized, Role: FOLLOWER
I20260811 07:06:59.282934 8078 ts_tablet_manager.cc:1939] T c195997ca5f74edf9939ca522bf1bfed P 9f2f0a6b9117448793b12c2d15173a45: Deleting consensus metadata
I20260811 07:06:59.282960 8100 consensus_queue.cc:260] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.283078 8100 ts_tablet_manager.cc:1428] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:06:59.283209 8098 tablet_bootstrap.cc:492] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: No bootstrap required, opened a new log
I20260811 07:06:59.283252 8098 ts_tablet_manager.cc:1397] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:06:59.283236 6951 catalog_manager.cc:4841] TS 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873): tablet c195997ca5f74edf9939ca522bf1bfed (table test-table [id=7193dc41312b4204a23099ccbe6b1b78]) successfully deleted
I20260811 07:06:59.283392 8098 raft_consensus.cc:348] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.283460 8098 raft_consensus.cc:374] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:06:59.283485 8098 raft_consensus.cc:729] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87abc8e64a4bc9ad365f6e774bb557, State: Initialized, Role: FOLLOWER
I20260811 07:06:59.283536 8098 consensus_queue.cc:260] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.283648 8098 ts_tablet_manager.cc:1428] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 07:06:59.285831 8113 raft_consensus.cc:482] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:06:59.285885 8113 raft_consensus.cc:504] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.286021 8113 leader_election.cc:288] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:59.286168 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965efe0c30a840b79b1215797794d326" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" is_pre_election: true
I20260811 07:06:59.286239 7515 raft_consensus.cc:2454] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:59.286329 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965efe0c30a840b79b1215797794d326" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" is_pre_election: true
I20260811 07:06:59.286361 7666 leader_election.cc:302] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d87abc8e64a4bc9ad365f6e774bb557, 9f2f0a6b9117448793b12c2d15173a45; no voters:
I20260811 07:06:59.286407 7256 raft_consensus.cc:2454] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 0.
I20260811 07:06:59.286438 8113 raft_consensus.cc:2781] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:06:59.286473 8113 raft_consensus.cc:482] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:06:59.286494 8113 raft_consensus.cc:3037] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.287149 8113 raft_consensus.cc:504] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.287292 8113 leader_election.cc:288] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Requested vote from peers adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799), 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163)
I20260811 07:06:59.287422 7515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965efe0c30a840b79b1215797794d326" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d87abc8e64a4bc9ad365f6e774bb557"
I20260811 07:06:59.287434 7256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965efe0c30a840b79b1215797794d326" candidate_uuid: "9f2f0a6b9117448793b12c2d15173a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1"
I20260811 07:06:59.287513 7515 raft_consensus.cc:3037] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.287513 7256 raft_consensus.cc:3037] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:06:59.288231 7256 raft_consensus.cc:2454] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:59.288226 7515 raft_consensus.cc:2454] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2f0a6b9117448793b12c2d15173a45 in term 1.
I20260811 07:06:59.288416 7671 leader_election.cc:302] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2f0a6b9117448793b12c2d15173a45, adfbe57f1eb2400b8eb4ef9890f22cd1; no voters:
I20260811 07:06:59.288553 8113 raft_consensus.cc:2781] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:06:59.288602 8113 raft_consensus.cc:686] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2f0a6b9117448793b12c2d15173a45, State: Running, Role: LEADER
I20260811 07:06:59.288652 8113 consensus_queue.cc:237] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } }
I20260811 07:06:59.289096 6972 catalog_manager.cc:5495] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "9f2f0a6b9117448793b12c2d15173a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2f0a6b9117448793b12c2d15173a45" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33873 } health_report { overall_health: HEALTHY } } }
I20260811 07:06:59.296048 7256 raft_consensus.cc:1261] T 965efe0c30a840b79b1215797794d326 P adfbe57f1eb2400b8eb4ef9890f22cd1 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:59.296069 7515 raft_consensus.cc:1261] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Refusing update from remote peer 9f2f0a6b9117448793b12c2d15173a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:06:59.296244 8113 consensus_queue.cc:1035] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d87abc8e64a4bc9ad365f6e774bb557" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:06:59.296420 8110 consensus_queue.cc:1035] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfbe57f1eb2400b8eb4ef9890f22cd1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:06:59.300498 8050 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 07:07:00.820149 6977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:00.820312 6977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:00.821173 6977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 066a8bd627704ebf84324d1c6e719436: Sending DeleteTablet for 3 replicas of tablet 965efe0c30a840b79b1215797794d326
I20260811 07:07:00.821458 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet 965efe0c30a840b79b1215797794d326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:00 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:33624
I20260811 07:07:00.821569 7478 tablet_service.cc:1514] Processing DeleteTablet for tablet 965efe0c30a840b79b1215797794d326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:00 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55038
I20260811 07:07:00.821684 8181 tablet_replica.cc:330] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: stopping tablet replica
I20260811 07:07:00.821774 8181 raft_consensus.cc:2229] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:00.821796 8182 tablet_replica.cc:330] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: stopping tablet replica
I20260811 07:07:00.821798 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7103
I20260811 07:07:00.821841 8181 raft_consensus.cc:2258] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:00.821854 8182 raft_consensus.cc:2229] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:00.830265 8182 raft_consensus.cc:2258] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:00.830444 8181 ts_tablet_manager.cc:1905] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:00.830592 8182 ts_tablet_manager.cc:1905] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:00.832644 8182 ts_tablet_manager.cc:1918] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:07:00.832643 8181 ts_tablet_manager.cc:1918] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:07:00.832722 8181 log.cc:1196] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/965efe0c30a840b79b1215797794d326
I20260811 07:07:00.832780 8182 log.cc:1196] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/965efe0c30a840b79b1215797794d326
I20260811 07:07:00.833037 8182 ts_tablet_manager.cc:1939] T 965efe0c30a840b79b1215797794d326 P 1d87abc8e64a4bc9ad365f6e774bb557: Deleting consensus metadata
I20260811 07:07:00.833034 8181 ts_tablet_manager.cc:1939] T 965efe0c30a840b79b1215797794d326 P 9f2f0a6b9117448793b12c2d15173a45: Deleting consensus metadata
I20260811 07:07:00.833437 6945 catalog_manager.cc:4841] TS 1d87abc8e64a4bc9ad365f6e774bb557 (127.3.236.130:43163): tablet 965efe0c30a840b79b1215797794d326 (table test-table [id=19951b7344474d4f89e2201c40eba523]) successfully deleted
I20260811 07:07:00.833487 6951 catalog_manager.cc:4841] TS 9f2f0a6b9117448793b12c2d15173a45 (127.3.236.131:33873): tablet 965efe0c30a840b79b1215797794d326 (table test-table [id=19951b7344474d4f89e2201c40eba523]) successfully deleted
W20260811 07:07:00.840350 6952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:44799 (error 108)
I20260811 07:07:00.840587 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7367
I20260811 07:07:00.848078 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:00.847744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44799 (local address 127.0.0.1:34174)
0811 07:07:00.847959 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:00.848016 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:44799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
W20260811 07:07:00.848243 6952 catalog_manager.cc:4584] TS adfbe57f1eb2400b8eb4ef9890f22cd1 (127.3.236.129:44799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 965efe0c30a840b79b1215797794d326: Network error: Client connection negotiation failed: client connection to 127.3.236.129:44799: connect: Connection refused (error 111)
I20260811 07:07:00.852998 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7640
I20260811 07:07:00.865105 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6921
2026-08-11T07:07:00Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6027 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:00 dist-test-slave-2d19 krb5kdc[8190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:00 dist-test-slave-2d19 krb5kdc[8190](info): set up 2 sockets
Aug 11 07:07:00 dist-test-slave-2d19 krb5kdc[8190](info): commencing operation
krb5kdc: starting...
W20260811 07:07:02.186952 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:02.315538 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38813
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:38813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:02.399509 8217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:02.399722 8217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:02.399745 8217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:02.400928 8217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:02.400969 8217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:02.400985 8217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:02.400998 8217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:02.401052 8217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:02.402498 8217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:38813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:02.402760 8217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:02.403004 8217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:02.408245 8229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.408293 8226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:02.408303 8217 server_base.cc:1034] running on GCE node
W20260811 07:07:02.408249 8232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:02.408756 8217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:02.409054 8217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:02.410208 8217 hybrid_clock.cc:648] HybridClock initialized: now 1786432022410195 us; error 28 us; skew 500 ppm
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:02.412729 8217 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:02.413101 8217 webserver.cc:460] Webserver started at http://127.3.236.190:35967/ using document root <none> and password file <none>
I20260811 07:07:02.413291 8217 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:02.413331 8217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:02.413417 8217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:02.414408 8217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "96059ff8f21f454c890e44bdddc72fd0"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "4a787904fd95f0664f5ecbde739ade90"
server_key_iv: "b57bf00a4e3ebdc845607b2ece443456"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.414721 8217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "96059ff8f21f454c890e44bdddc72fd0"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "4a787904fd95f0664f5ecbde739ade90"
server_key_iv: "b57bf00a4e3ebdc845607b2ece443456"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.415982 8217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:07:02.416739 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.416940 8217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:02.417032 8217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "96059ff8f21f454c890e44bdddc72fd0"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "4a787904fd95f0664f5ecbde739ade90"
server_key_iv: "b57bf00a4e3ebdc845607b2ece443456"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.417086 8217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:02.461335 8217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:02.463251 8217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:02.463368 8217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:02.467564 8217 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:38813
I20260811 07:07:02.467631 8346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:38813 every 8 connection(s)
I20260811 07:07:02.468015 8217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:02.469012 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:02.471226 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8217
I20260811 07:07:02.471303 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:02.471458 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0: Bootstrap starting.
I20260811 07:07:02.471493 4018 external_mini_cluster.cc:1428] Setting key 6052532ed7bfda4c6574e1f459b0f4ba
I20260811 07:07:02.472160 8348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:02.472545 8348 log.cc:824] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:02.473237 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0: No bootstrap required, opened a new log
I20260811 07:07:02.475046 8348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } }
I20260811 07:07:02.475217 8348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:02.475281 8348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96059ff8f21f454c890e44bdddc72fd0, State: Initialized, Role: FOLLOWER
I20260811 07:07:02.475453 8348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } }
I20260811 07:07:02.475550 8348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:02.475615 8348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:02.475668 8348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:02.476190 8348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } }
I20260811 07:07:02.476336 8348 leader_election.cc:302] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96059ff8f21f454c890e44bdddc72fd0; no voters:
I20260811 07:07:02.476514 8348 leader_election.cc:288] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:02.476719 8365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:02.476785 8348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:02.476893 8365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 96059ff8f21f454c890e44bdddc72fd0, State: Running, Role: LEADER
I20260811 07:07:02.477020 8365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } }
I20260811 07:07:02.477440 8365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96059ff8f21f454c890e44bdddc72fd0. Latest consensus state: current_term: 1 leader_uuid: "96059ff8f21f454c890e44bdddc72fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } } }
I20260811 07:07:02.477550 8365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:02.477871 8375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:02.477900 8364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96059ff8f21f454c890e44bdddc72fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96059ff8f21f454c890e44bdddc72fd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38813 } } }
I20260811 07:07:02.477993 8364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0 [sys.catalog]: This master's current role is: LEADER
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:02.478842 8375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:02.480618 8375 catalog_manager.cc:1347] Generated new cluster ID: e810254ebfab437eb8c4ccd6fa8caee3
I20260811 07:07:02.480675 8375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:02.482895 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:02.472075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.3.236.190:38813)
0811 07:07:02.472314 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:02.472324 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:02.472359 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:07:02.472880 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:02.472888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:02.472938 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:02.473099 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:02.473825 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.474721 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.475610 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.476078 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.476218 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:02.479319 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:02.479323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:02.479323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:02.479337 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:02.481329 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.481706 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.481710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.481710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.481815 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.482017 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.482018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.482018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.482258 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:02.482450 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:02.482566 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 07:07:02.482720 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260811 07:07:02.496043 8375 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:02.496833 8375 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:02.501572 8375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96059ff8f21f454c890e44bdddc72fd0: Generated new TSK 0
I20260811 07:07:02.501757 8375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:02.535646 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:02.629223 8393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:02.629468 8393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:02.629506 8393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:02.630673 8393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:02.630739 8393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:02.630785 8393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:02.632441 8393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:02.632719 8393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:02.632961 8393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:02.638144 8402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.638187 8400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.638154 8404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:02.638394 8393 server_base.cc:1034] running on GCE node
I20260811 07:07:02.638584 8393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:02.638835 8393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:02.639976 8393 hybrid_clock.cc:648] HybridClock initialized: now 1786432022639964 us; error 28 us; skew 500 ppm
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:02.642649 8393 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:02.643203 8393 webserver.cc:460] Webserver started at http://127.3.236.129:35105/ using document root <none> and password file <none>
I20260811 07:07:02.643445 8393 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:02.643512 8393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:02.643668 8393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:02.644805 8393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "fd58a70d8b2a461084a9cc60e5a806d1"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "2306a8b62011d65504387473c5a1f30a"
server_key_iv: "478996f3a9f747559c0ee08519e23d7f"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.645184 8393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "fd58a70d8b2a461084a9cc60e5a806d1"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "2306a8b62011d65504387473c5a1f30a"
server_key_iv: "478996f3a9f747559c0ee08519e23d7f"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.646550 8393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:02.647255 8415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.647428 8393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:02.647534 8393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "fd58a70d8b2a461084a9cc60e5a806d1"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "2306a8b62011d65504387473c5a1f30a"
server_key_iv: "478996f3a9f747559c0ee08519e23d7f"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.647614 8393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:02.669389 8393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:02.671868 8393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:02.672046 8393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:02.672358 8393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:02.672701 8393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:02.672756 8393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.672794 8393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:02.672820 8393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.679425 8393 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:36607
I20260811 07:07:02.679471 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:36607 every 8 connection(s)
I20260811 07:07:02.679803 8393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:02.682006 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8393
I20260811 07:07:02.682119 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:02.682307 4018 external_mini_cluster.cc:1428] Setting key 092c829c0a3bfc7f2e125e59ef8bd920
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:02.690207 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:02.680564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:46321 (local address 127.3.236.190:38813)
0811 07:07:02.680610 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:02.680612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:02.680888 (+ 276us) server_negotiation.cc:407] Connection header received
0811 07:07:02.681429 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:02.681431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:02.681444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:02.681507 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:02.682829 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.683362 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.684446 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.684748 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.684826 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:02.686996 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:02.686998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:02.686998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:02.687006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:02.688795 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.689206 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.689207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.689207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.689275 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.689524 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.689525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.689525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.689669 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:02.689746 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:02.689989 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 07:07:02.690080 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:07:02.690243 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:02.680562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38813 (local address 127.3.236.129:46321)
0811 07:07:02.680753 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:02.680790 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:02.681321 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:02.681587 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:02.681601 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:02.682425 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:02.682731 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:02.682737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.683468 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.683469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:02.684360 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:02.684363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.684835 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.684836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:02.684953 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:02.685497 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:02.685506 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:02.686896 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:02.688882 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:02.688889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:02.688891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:02.689113 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:02.689350 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:02.689351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:02.689351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:02.689452 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:02.689739 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:02.689744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:02.689855 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 07:07:02.689942 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:07:02.690060 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260811 07:07:02.690806 8630 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38813
I20260811 07:07:02.690917 8630 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:02.691126 8630 heartbeater.cc:511] Master 127.3.236.190:38813 requested a full tablet report, sending...
I20260811 07:07:02.691687 8281 ts_manager.cc:194] Registered new tserver with Master: fd58a70d8b2a461084a9cc60e5a806d1 (127.3.236.129:36607)
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:07:02.692477 8281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:46321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:02.711459 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:02.795166 8638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:02.795369 8638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:02.795392 8638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:02.796578 8638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:02.796625 8638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:02.796653 8638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:02.798123 8638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:02.798394 8638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:02.798660 8638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:02.803779 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.803930 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.803791 8646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:02.804173 8638 server_base.cc:1034] running on GCE node
I20260811 07:07:02.804481 8638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:02.804757 8638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:02.805986 8638 hybrid_clock.cc:648] HybridClock initialized: now 1786432022805958 us; error 60 us; skew 500 ppm
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:02.808889 8638 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:02.809477 8638 webserver.cc:460] Webserver started at http://127.3.236.130:45633/ using document root <none> and password file <none>
I20260811 07:07:02.809746 8638 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:02.809824 8638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:02.809998 8638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:02.811151 8638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "b854f4c6bad0458aacf338b66c29e932"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "3bb4b776a1cbb938fe7d97bf278657ca"
server_key_iv: "873b71ca4c49614943ffc4ea05ec64e5"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.811564 8638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "b854f4c6bad0458aacf338b66c29e932"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "3bb4b776a1cbb938fe7d97bf278657ca"
server_key_iv: "873b71ca4c49614943ffc4ea05ec64e5"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.813194 8638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:02.814069 8663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.814275 8638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:02.814392 8638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "b854f4c6bad0458aacf338b66c29e932"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "3bb4b776a1cbb938fe7d97bf278657ca"
server_key_iv: "873b71ca4c49614943ffc4ea05ec64e5"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.814476 8638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:02.830065 8638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:02.832129 8638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:02.832386 8638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:02.832636 8638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:02.832970 8638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:02.833027 8638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.833069 8638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:02.833096 8638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.839419 8638 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:45859
I20260811 07:07:02.839500 8873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:45859 every 8 connection(s)
I20260811 07:07:02.839810 8638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:02.847591 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8638
I20260811 07:07:02.847708 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:02.847970 4018 external_mini_cluster.cc:1428] Setting key 119e9d5c8be19312d457bd950dac7de0
I20260811 07:07:02.850023 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:02.840703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:60489 (local address 127.3.236.190:38813)
0811 07:07:02.840774 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:02.840777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:02.840956 (+ 179us) server_negotiation.cc:407] Connection header received
0811 07:07:02.841456 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:02.841457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:02.841469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:02.841534 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:02.842595 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.843118 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.844227 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.844511 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.844572 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:02.846820 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:02.846821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:02.846822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:02.846830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:02.848562 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.848991 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.848993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.848993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.849069 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:02.849334 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:02.849335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:02.849335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:02.849462 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:02.849540 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:02.849828 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 07:07:02.849918 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:07:02.850238 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:02.840628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38813 (local address 127.3.236.130:60489)
0811 07:07:02.840817 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:02.840859 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:02.841353 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:02.841619 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:02.841622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:02.842267 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:02.842479 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:02.842484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.843242 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.843243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:02.844130 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:02.844133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:02.844611 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:02.844612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:02.844724 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:02.845207 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:02.845216 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:02.846646 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:02.848669 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:02.848684 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:02.848686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:02.848905 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:02.849148 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:02.849149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:02.849149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:02.849264 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:02.849532 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:02.849540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:02.849679 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 07:07:02.849791 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 07:07:02.849959 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260811 07:07:02.850876 8874 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38813
I20260811 07:07:02.850977 8874 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:02.851179 8874 heartbeater.cc:511] Master 127.3.236.190:38813 requested a full tablet report, sending...
I20260811 07:07:02.851622 8281 ts_manager.cc:194] Registered new tserver with Master: b854f4c6bad0458aacf338b66c29e932 (127.3.236.130:45859)
I20260811 07:07:02.852013 8281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:60489
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:02.876084 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:02.962402 8881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:02.962656 8881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:02.962695 8881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:02.963888 8881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:02.963963 8881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:02.964010 8881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:02.965615 8881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:02.965904 8881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:02.966161 8881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:02.971472 8889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.971586 8892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:02.971462 8895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:02.971768 8881 server_base.cc:1034] running on GCE node
I20260811 07:07:02.972028 8881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:02.972352 8881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:02.973592 8881 hybrid_clock.cc:648] HybridClock initialized: now 1786432022973575 us; error 26 us; skew 500 ppm
Aug 11 07:07:02 dist-test-slave-2d19 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:02.976174 8881 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:02.976716 8881 webserver.cc:460] Webserver started at http://127.3.236.131:38955/ using document root <none> and password file <none>
I20260811 07:07:02.976982 8881 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:02.977054 8881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:02.977238 8881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:02.978497 8881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "9b636e349fb44b9cabb2097b5f63e77e"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "88e4951c3dfd2355e922754f70da7dcf"
server_key_iv: "1989dd7be25e150ff856c312870fa20d"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.978899 8881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "9b636e349fb44b9cabb2097b5f63e77e"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "88e4951c3dfd2355e922754f70da7dcf"
server_key_iv: "1989dd7be25e150ff856c312870fa20d"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.980224 8881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:07:02.981070 8903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:02.981266 8881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:02.981382 8881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "9b636e349fb44b9cabb2097b5f63e77e"
format_stamp: "Formatted at 2026-08-11 07:07:02 on dist-test-slave-2d19"
server_key: "88e4951c3dfd2355e922754f70da7dcf"
server_key_iv: "1989dd7be25e150ff856c312870fa20d"
server_key_version: "encryptionkey@0"
I20260811 07:07:02.981462 8881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:03.000937 8881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:03.003618 8881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:03.003780 8881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:03.004048 8881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:03.004437 8881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:03.004475 8881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:03.004498 8881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:03.004541 8881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:03.010504 8881 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:45249
I20260811 07:07:03.010586 9109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:45249 every 8 connection(s)
I20260811 07:07:03.010924 8881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:03.011858 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8881
I20260811 07:07:03.012010 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:03.012230 4018 external_mini_cluster.cc:1428] Setting key a2cebf3617d7097fc3085f655af057e5
Aug 11 07:07:03 dist-test-slave-2d19 krb5kdc[8190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:03.020377 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:03.011717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:42731 (local address 127.3.236.190:38813)
0811 07:07:03.011791 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:03.011794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:03.012032 (+ 238us) server_negotiation.cc:407] Connection header received
0811 07:07:03.012781 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:03.012782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:03.012799 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:03.012870 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:03.013948 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:03.014438 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:03.015394 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:03.015570 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:03.015622 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:03.017784 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:03.017785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:03.017786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:03.017792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:03.019074 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:03.019444 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:03.019445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:03.019445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:03.019508 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:03.019741 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:03.019741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:03.019742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:03.019875 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:03.019936 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:03.020189 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 07:07:03.020291 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:03.020543 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:03.011661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38813 (local address 127.3.236.131:42731)
0811 07:07:03.011869 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:03.011930 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:03.012669 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:03.012960 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:03.012965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:03.013647 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:03.013840 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:03.013845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:03.014519 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:03.014520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:03.015322 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:03.015324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:03.015642 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:03.015643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:03.015745 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:03.016193 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:03.016202 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:03.017702 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:03.019152 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:03.019158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:03.019161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:03.019370 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:03.019579 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:03.019580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:03.019580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:03.019680 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:03.019966 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:03.019971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:03.020080 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 07:07:03.020161 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:07:03.020334 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260811 07:07:03.021034 9111 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38813
I20260811 07:07:03.021116 9111 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:03.021257 9111 heartbeater.cc:511] Master 127.3.236.190:38813 requested a full tablet report, sending...
I20260811 07:07:03.021556 8281 ts_manager.cc:194] Registered new tserver with Master: 9b636e349fb44b9cabb2097b5f63e77e (127.3.236.131:45249)
I20260811 07:07:03.021888 8281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:42731
I20260811 07:07:03.024006 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:07:03.037635 8359 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:03.036740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.3.236.190:38813)
0811 07:07:03.036810 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:03.036812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:03.036909 (+ 97us) server_negotiation.cc:407] Connection header received
0811 07:07:03.036965 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:03.036966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:03.036975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:03.037052 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:03.037581 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37504: BlockingRecv error: recv got EOF from 127.0.0.1:37504 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:07:03.038725 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8393
I20260811 07:07:03.062244 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8638
I20260811 07:07:03.072846 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8881
I20260811 07:07:03.088800 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8217
2026-08-11T07:07:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2226 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:03 dist-test-slave-2d19 krb5kdc[9132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:03 dist-test-slave-2d19 krb5kdc[9132](info): set up 2 sockets
Aug 11 07:07:03 dist-test-slave-2d19 krb5kdc[9132](info): commencing operation
krb5kdc: starting...
W20260811 07:07:04.377522 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:04.512549 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:33611
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:33611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:04.596112 9153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:04.596382 9153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:04.596422 9153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:04.597613 9153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:04.597659 9153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:04.597680 9153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:04.597708 9153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:04.597738 9153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:04.599239 9153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:33611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:33611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:04.599509 9153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:04.599756 9153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:04.604856 9162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:04.604923 9161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:04.604936 9153 server_base.cc:1034] running on GCE node
W20260811 07:07:04.604857 9165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:04.605428 9153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:04.605755 9153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:04.606986 9153 hybrid_clock.cc:648] HybridClock initialized: now 1786432024606965 us; error 42 us; skew 500 ppm
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432024, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:04.609917 9153 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:04.610451 9153 webserver.cc:460] Webserver started at http://127.3.236.190:35983/ using document root <none> and password file <none>
I20260811 07:07:04.610705 9153 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:04.610781 9153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:04.610952 9153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:04.612121 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "30fb918b8b31457daa09ee4572fd7fe3"
format_stamp: "Formatted at 2026-08-11 07:07:04 on dist-test-slave-2d19"
server_key: "994123ffdbe06cf8e7a6f8c0f86fd8cf"
server_key_iv: "77131933999abffd5eac8a719c967210"
server_key_version: "encryptionkey@0"
I20260811 07:07:04.612636 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "30fb918b8b31457daa09ee4572fd7fe3"
format_stamp: "Formatted at 2026-08-11 07:07:04 on dist-test-slave-2d19"
server_key: "994123ffdbe06cf8e7a6f8c0f86fd8cf"
server_key_iv: "77131933999abffd5eac8a719c967210"
server_key_version: "encryptionkey@0"
I20260811 07:07:04.614095 9153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:04.614916 9177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:04.615091 9153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:04.615206 9153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "30fb918b8b31457daa09ee4572fd7fe3"
format_stamp: "Formatted at 2026-08-11 07:07:04 on dist-test-slave-2d19"
server_key: "994123ffdbe06cf8e7a6f8c0f86fd8cf"
server_key_iv: "77131933999abffd5eac8a719c967210"
server_key_version: "encryptionkey@0"
I20260811 07:07:04.615288 9153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:04.643399 9153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:04.645618 9153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:04.645802 9153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:04.650100 9153 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:33611
I20260811 07:07:04.650143 9261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:33611 every 8 connection(s)
I20260811 07:07:04.650532 9153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:04.651401 9262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:04.653683 9262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3: Bootstrap starting.
I20260811 07:07:04.654237 9262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:04.654476 9262 log.cc:824] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:04.655148 9262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3: No bootstrap required, opened a new log
I20260811 07:07:04.656769 9262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } }
I20260811 07:07:04.656909 9262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:04.656932 9262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30fb918b8b31457daa09ee4572fd7fe3, State: Initialized, Role: FOLLOWER
I20260811 07:07:04.657058 9262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } }
I20260811 07:07:04.657140 9262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:04.657161 9262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:04.657188 9262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:04.657914 9262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } }
I20260811 07:07:04.657998 9262 leader_election.cc:302] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30fb918b8b31457daa09ee4572fd7fe3; no voters:
I20260811 07:07:04.658128 9262 leader_election.cc:288] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:04.658208 9272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:04.658390 9272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [term 1 LEADER]: Becoming Leader. State: Replica: 30fb918b8b31457daa09ee4572fd7fe3, State: Running, Role: LEADER
I20260811 07:07:04.658401 9262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:04.658496 9272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } }
I20260811 07:07:04.658555 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9153
I20260811 07:07:04.658638 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:04.658820 4018 external_mini_cluster.cc:1428] Setting key b36b09d5f1ca46d2cd8cd2ead245f2e5
I20260811 07:07:04.658839 9276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30fb918b8b31457daa09ee4572fd7fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } } }
I20260811 07:07:04.658846 9277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30fb918b8b31457daa09ee4572fd7fe3. Latest consensus state: current_term: 1 leader_uuid: "30fb918b8b31457daa09ee4572fd7fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30fb918b8b31457daa09ee4572fd7fe3" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33611 } } }
I20260811 07:07:04.658946 9277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:04.658931 9276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:04.659183 9282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:04.660183 9282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:04.662004 9282 catalog_manager.cc:1347] Generated new cluster ID: 541b9632203d4c74b0bfbbb27327c31e
I20260811 07:07:04.662060 9282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:04.670830 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:04.659432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33502 (local address 127.3.236.190:33611)
0811 07:07:04.659614 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:04.659624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:04.659715 (+ 91us) server_negotiation.cc:407] Connection header received
0811 07:07:04.659906 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:04.659914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:04.659976 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:04.661092 (+ 1116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:04.661188 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:04.662509 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:04.663605 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:04.664198 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:04.664345 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:04.667134 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:04.667138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:04.667138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:04.667152 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:04.669169 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:04.669576 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:04.669580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:04.669580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:04.669700 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:04.669914 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:04.669915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:04.669916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:04.670226 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:04.670387 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:04.670463 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 07:07:04.670616 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260811 07:07:04.672052 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:07:04.672212 9282 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:04.672921 9282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:04.681250 9282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30fb918b8b31457daa09ee4572fd7fe3: Generated new TSK 0
I20260811 07:07:04.681447 9282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:04.688432 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:04.683877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.3.236.190:33611)
0811 07:07:04.683951 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:04.683953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:04.684098 (+ 145us) server_negotiation.cc:407] Connection header received
0811 07:07:04.684129 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:04.684130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:04.684142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:04.684214 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:04.684633 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:04.685925 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:04.686667 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:04.687003 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:04.687077 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:04.687872 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:04.687873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:04.687874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:04.688094 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:04.688112 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:04.688120 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:04.688184 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:04.688326 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 07:07:04.688352 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:04.690851 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9153
2026-08-11T07:07:04Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1603 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9326](info): set up 2 sockets
Aug 11 07:07:04 dist-test-slave-2d19 krb5kdc[9326](info): commencing operation
krb5kdc: starting...
W20260811 07:07:05.987067 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:06.115859 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:45151
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:06.200882 9347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:06.201092 9347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:06.201113 9347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:06.202248 9347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:06.202291 9347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:06.202306 9347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:06.202318 9347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:06.202370 9347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:06.203814 9347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:45151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:06.204079 9347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:06.204378 9347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:06.209331 9356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.209352 9347 server_base.cc:1034] running on GCE node
W20260811 07:07:06.209324 9357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.209319 9359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.209842 9347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:06.210151 9347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:06.211297 9347 hybrid_clock.cc:648] HybridClock initialized: now 1786432026211284 us; error 28 us; skew 500 ppm
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:06.213884 9347 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:06.214344 9347 webserver.cc:460] Webserver started at http://127.3.236.190:45421/ using document root <none> and password file <none>
I20260811 07:07:06.214568 9347 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:06.214638 9347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:06.214790 9347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:06.215873 9347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "254040d9be544c3abeebcbe57eeabf64"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "24db13ae7068bc149dad1b29b7c41dd0"
server_key_iv: "69cb4cfd306996c0aac5856fc4744494"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.216295 9347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "254040d9be544c3abeebcbe57eeabf64"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "24db13ae7068bc149dad1b29b7c41dd0"
server_key_iv: "69cb4cfd306996c0aac5856fc4744494"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.217701 9347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:06.218453 9370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.218643 9347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:06.218746 9347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "254040d9be544c3abeebcbe57eeabf64"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "24db13ae7068bc149dad1b29b7c41dd0"
server_key_iv: "69cb4cfd306996c0aac5856fc4744494"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.218837 9347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:06.227826 9347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:06.229806 9347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:06.229959 9347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:06.234496 9347 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:45151
I20260811 07:07:06.234537 9458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:45151 every 8 connection(s)
I20260811 07:07:06.234889 9347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:06.236001 9460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:06.238307 9460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: Bootstrap starting.
I20260811 07:07:06.238893 9460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:06.239180 9460 log.cc:824] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:06.239867 9460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: No bootstrap required, opened a new log
I20260811 07:07:06.241081 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9347
I20260811 07:07:06.241182 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:06.241353 4018 external_mini_cluster.cc:1428] Setting key 0ef139845a42963eb78731039dee37fa
I20260811 07:07:06.242067 9460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } }
I20260811 07:07:06.242204 9460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:06.242228 9460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254040d9be544c3abeebcbe57eeabf64, State: Initialized, Role: FOLLOWER
I20260811 07:07:06.242365 9460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } }
I20260811 07:07:06.242460 9460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:06.242528 9460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:06.242660 9460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:06.243233 9460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } }
I20260811 07:07:06.243355 9460 leader_election.cc:302] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 254040d9be544c3abeebcbe57eeabf64; no voters:
I20260811 07:07:06.243556 9460 leader_election.cc:288] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:06.243585 9471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:06.243696 9471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [term 1 LEADER]: Becoming Leader. State: Replica: 254040d9be544c3abeebcbe57eeabf64, State: Running, Role: LEADER
I20260811 07:07:06.243830 9460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:06.243847 9471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } }
I20260811 07:07:06.244170 9471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 254040d9be544c3abeebcbe57eeabf64. Latest consensus state: current_term: 1 leader_uuid: "254040d9be544c3abeebcbe57eeabf64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } } }
I20260811 07:07:06.244182 9473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "254040d9be544c3abeebcbe57eeabf64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254040d9be544c3abeebcbe57eeabf64" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45151 } } }
I20260811 07:07:06.244339 9471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:06.244352 9473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64 [sys.catalog]: This master's current role is: LEADER
W20260811 07:07:06.246129 9499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:07:06.247330 9499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:07:06.247433 9479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:06.247624 9479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:06.249892 9479 catalog_manager.cc:1347] Generated new cluster ID: e47540fa44f344b281e54401e450f9ea
I20260811 07:07:06.249943 9479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:06.255594 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.241893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.3.236.190:45151)
0811 07:07:06.242085 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:06.242097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:06.242127 (+ 30us) server_negotiation.cc:407] Connection header received
0811 07:07:06.242241 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:06.242249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:06.242306 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:06.242455 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:06.243305 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.244831 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.245942 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.246472 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.246607 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.249465 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:06.249468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:06.249469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:06.249479 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:06.253881 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.254332 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.254335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.254336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.254454 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.254688 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.254689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.254689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.254984 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:06.255152 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:06.255267 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 07:07:06.255407 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260811 07:07:06.256127 9479 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:06.256695 9479 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:06.261394 9479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 254040d9be544c3abeebcbe57eeabf64: Generated new TSK 0
I20260811 07:07:06.261566 9479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:06.291318 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:06.375856 9514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:06.376123 9514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:06.376156 9514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:06.377360 9514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:06.377404 9514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:06.377434 9514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:06.378816 9514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:06.379006 9514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:06.379182 9514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:06.384341 9527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.384217 9524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.384487 9514 server_base.cc:1034] running on GCE node
W20260811 07:07:06.384229 9523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.384807 9514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:06.385057 9514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:06.386209 9514 hybrid_clock.cc:648] HybridClock initialized: now 1786432026386195 us; error 28 us; skew 500 ppm
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:06.388855 9514 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:06.389329 9514 webserver.cc:460] Webserver started at http://127.3.236.129:34277/ using document root <none> and password file <none>
I20260811 07:07:06.389593 9514 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:06.389673 9514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:06.389828 9514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:06.390908 9514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "22d60933b9a94ef88b91932e3f6c76d1"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "49c83f7fdb1ff69c449490e8e6327db7"
server_key_iv: "b5f8f850dfa2d92599f1ccf4e368837c"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.391285 9514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "22d60933b9a94ef88b91932e3f6c76d1"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "49c83f7fdb1ff69c449490e8e6327db7"
server_key_iv: "b5f8f850dfa2d92599f1ccf4e368837c"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.392726 9514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:06.393486 9540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.393671 9514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:06.393767 9514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "22d60933b9a94ef88b91932e3f6c76d1"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "49c83f7fdb1ff69c449490e8e6327db7"
server_key_iv: "b5f8f850dfa2d92599f1ccf4e368837c"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.393851 9514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:06.419494 9514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:06.421666 9514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:06.421842 9514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:06.422117 9514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:06.422509 9514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:06.422567 9514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.422611 9514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:06.422641 9514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.428846 9514 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:39499
I20260811 07:07:06.428923 9725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:39499 every 8 connection(s)
I20260811 07:07:06.429270 9514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:06.436852 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9514
I20260811 07:07:06.436966 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:06.437208 4018 external_mini_cluster.cc:1428] Setting key 63e21555f135dcb66ebebac2cc18579d
I20260811 07:07:06.438931 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.430078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:48415 (local address 127.3.236.190:45151)
0811 07:07:06.430146 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:06.430148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:06.430338 (+ 190us) server_negotiation.cc:407] Connection header received
0811 07:07:06.430853 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:06.430854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:06.430868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:06.430925 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:06.431856 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.432455 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.433463 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.433658 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.433709 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.435736 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:06.435737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:06.435738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:06.435744 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:06.437298 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.437781 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.437783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.437783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.437877 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.438251 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.438252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.438252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.438367 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:06.438428 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:06.438792 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 07:07:06.438866 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:07:06.439213 9729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.430035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45151 (local address 127.3.236.129:48415)
0811 07:07:06.430215 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:06.430258 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:06.430762 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:06.430992 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:06.430996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:06.431568 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:06.431769 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.431774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.432568 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.432569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.433385 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.433387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.433734 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.433735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.433838 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.434272 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:06.434281 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:06.435647 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:06.437411 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.437420 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.437423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.437682 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.437994 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.437995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.437995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.438169 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.438441 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:06.438448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:06.438618 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 07:07:06.438740 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 07:07:06.438928 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260811 07:07:06.439736 9726 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45151
I20260811 07:07:06.439860 9726 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:06.440096 9726 heartbeater.cc:511] Master 127.3.236.190:45151 requested a full tablet report, sending...
I20260811 07:07:06.440666 9400 ts_manager.cc:194] Registered new tserver with Master: 22d60933b9a94ef88b91932e3f6c76d1 (127.3.236.129:39499)
I20260811 07:07:06.441282 9400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:48415
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:06.464545 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:06.548843 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:06.549091 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:06.549130 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:06.550396 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:06.550462 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:06.550503 9734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:06.552035 9734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:06.552390 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:06.552636 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:06.557878 9746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.557883 9748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.557921 9745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.558266 9734 server_base.cc:1034] running on GCE node
I20260811 07:07:06.558509 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:06.558759 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:06.559898 9734 hybrid_clock.cc:648] HybridClock initialized: now 1786432026559889 us; error 33 us; skew 500 ppm
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:06.562489 9734 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:06.562876 9734 webserver.cc:460] Webserver started at http://127.3.236.130:33285/ using document root <none> and password file <none>
I20260811 07:07:06.563069 9734 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:06.563117 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:06.563208 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:06.564215 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "1737a6506a6f4325a2fc352aff7a1c75"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "af6f6690dd8223293ed3c8f50ccb2594"
server_key_iv: "6e74fe49cb4a70d22b5235c5f408f6e4"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.564599 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "1737a6506a6f4325a2fc352aff7a1c75"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "af6f6690dd8223293ed3c8f50ccb2594"
server_key_iv: "6e74fe49cb4a70d22b5235c5f408f6e4"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.565855 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:06.566576 9756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.566773 9734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:06.566881 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "1737a6506a6f4325a2fc352aff7a1c75"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "af6f6690dd8223293ed3c8f50ccb2594"
server_key_iv: "6e74fe49cb4a70d22b5235c5f408f6e4"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.566956 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:06.580235 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:06.582566 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:06.582724 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:06.582947 9734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:06.583274 9734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:06.583326 9734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.583369 9734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:06.583407 9734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.589718 9734 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:43649
I20260811 07:07:06.589795 9972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:43649 every 8 connection(s)
I20260811 07:07:06.590097 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:06.599105 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.590814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45151 (local address 127.3.236.130:32879)
0811 07:07:06.590981 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:06.591022 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:06.591508 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:06.591716 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:06.591720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:06.592364 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:06.592555 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.592561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.593217 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.593218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.593994 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.593996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.594327 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.594328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.594428 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.594969 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:06.594977 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:06.596378 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:06.597910 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.597917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.597918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.598134 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.598320 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.598320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.598321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.598415 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.598630 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:06.598635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:06.598743 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 07:07:06.598823 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 07:07:06.598936 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260811 07:07:06.599129 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.590831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:32879 (local address 127.3.236.190:45151)
0811 07:07:06.590899 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:06.590901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:06.591111 (+ 210us) server_negotiation.cc:407] Connection header received
0811 07:07:06.591590 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:06.591591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:06.591600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:06.591663 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:06.592641 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.593136 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.594070 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.594255 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.594304 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.596466 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:06.596467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:06.596467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:06.596474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:06.597771 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.598202 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.598203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.598203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.598259 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.598470 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.598471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.598471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.598575 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:06.598627 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:06.598985 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 07:07:06.599064 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:07:06.599747 9975 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45151
I20260811 07:07:06.599862 9975 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:06.600047 9975 heartbeater.cc:511] Master 127.3.236.190:45151 requested a full tablet report, sending...
I20260811 07:07:06.600097 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9734
I20260811 07:07:06.600209 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:06.600435 4018 external_mini_cluster.cc:1428] Setting key 85454cbaf7a8090314f9e2df26e10fbe
I20260811 07:07:06.600505 9400 ts_manager.cc:194] Registered new tserver with Master: 1737a6506a6f4325a2fc352aff7a1c75 (127.3.236.130:43649)
I20260811 07:07:06.600947 9400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:32879
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:06.626425 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:06.708657 9981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:06.708853 9981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:06.708875 9981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:06.709935 9981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:06.709970 9981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:06.709996 9981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:06.711376 9981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:06.711558 9981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:06.711743 9981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:06.716737 9992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.716792 9990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:06.716732 9989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:06.717043 9981 server_base.cc:1034] running on GCE node
I20260811 07:07:06.717290 9981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:06.717571 9981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:06.718715 9981 hybrid_clock.cc:648] HybridClock initialized: now 1786432026718703 us; error 29 us; skew 500 ppm
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:06.721330 9981 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:06.721809 9981 webserver.cc:460] Webserver started at http://127.3.236.131:44217/ using document root <none> and password file <none>
I20260811 07:07:06.722074 9981 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:06.722146 9981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:06.722302 9981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:06.723412 9981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "5174caa1a39f492b9f614efd5b79a828"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "28a596077a2a79b84704d77406acaaa2"
server_key_iv: "4bd353c14dd32f7eb15f253752147a84"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.723783 9981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "5174caa1a39f492b9f614efd5b79a828"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "28a596077a2a79b84704d77406acaaa2"
server_key_iv: "4bd353c14dd32f7eb15f253752147a84"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.725275 9981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:06.725983 10003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.726181 9981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.726285 9981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "5174caa1a39f492b9f614efd5b79a828"
format_stamp: "Formatted at 2026-08-11 07:07:06 on dist-test-slave-2d19"
server_key: "28a596077a2a79b84704d77406acaaa2"
server_key_iv: "4bd353c14dd32f7eb15f253752147a84"
server_key_version: "encryptionkey@0"
I20260811 07:07:06.726368 9981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:06.738459 9981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:06.740509 9981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:06.740679 9981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:06.740931 9981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:06.741262 9981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:06.741319 9981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.741364 9981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:06.741410 9981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:06.748100 9981 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:44319
I20260811 07:07:06.748209 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:44319 every 8 connection(s)
I20260811 07:07:06.748538 9981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:06.751336 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9981
I20260811 07:07:06.751422 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:06.751639 4018 external_mini_cluster.cc:1428] Setting key 028fbc2d500053926d2efd5e2c868088
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:06.757988 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.749307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:49497 (local address 127.3.236.190:45151)
0811 07:07:06.749372 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:06.749374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:06.749589 (+ 215us) server_negotiation.cc:407] Connection header received
0811 07:07:06.750094 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:06.750094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:06.750105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:06.750166 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:06.751174 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.751798 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.753071 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.753254 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.753310 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.755451 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:06.755452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:06.755453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:06.755475 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:06.756840 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.757205 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.757206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.757207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.757270 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.757485 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.757486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.757486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.757588 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:06.757639 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:06.757864 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 07:07:06.757925 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:06.758145 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.749282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45151 (local address 127.3.236.131:49497)
0811 07:07:06.749450 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:06.749494 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:06.750008 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:06.750269 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:06.750273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:06.750894 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:06.751088 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.751094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.751888 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.751889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.752988 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:06.752991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.753329 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.753330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:06.753425 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.753968 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:06.753992 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:06.755367 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:06.756911 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.756918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.756920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.757131 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.757333 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:06.757333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:06.757334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:06.757430 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:06.757644 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:06.757648 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:06.757755 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 07:07:06.757836 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:07:06.757948 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260811 07:07:06.758610 10202 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45151
I20260811 07:07:06.758698 10202 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:06.758844 10202 heartbeater.cc:511] Master 127.3.236.190:45151 requested a full tablet report, sending...
I20260811 07:07:06.759212 9400 ts_manager.cc:194] Registered new tserver with Master: 5174caa1a39f492b9f614efd5b79a828 (127.3.236.131:44319)
I20260811 07:07:06.759584 9400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:49497
I20260811 07:07:06.763494 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[9326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432026, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:06.791916 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:06.780227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.3.236.190:45151)
0811 07:07:06.780351 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:06.780354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:06.780439 (+ 85us) server_negotiation.cc:407] Connection header received
0811 07:07:06.780541 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:06.780542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:06.780552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:06.780634 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:06.781202 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.781700 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.782466 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:06.782667 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:06.782750 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:06.789246 (+ 6496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:06.789248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:06.789248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:06.789256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:06.790663 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.791025 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.791026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.791026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.791088 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:06.791297 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:06.791298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:06.791298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:06.791488 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:06.791599 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:06.791713 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 07:07:06.791793 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 07:07:06.792218 9400 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52642
I20260811 07:07:06.792709 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9514
I20260811 07:07:06.804554 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9734
I20260811 07:07:06.818266 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9981
I20260811 07:07:06.832008 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9347
2026-08-11T07:07:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2134 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[10223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[10223](info): set up 2 sockets
Aug 11 07:07:06 dist-test-slave-2d19 krb5kdc[10223](info): commencing operation
krb5kdc: starting...
W20260811 07:07:08.104887 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:08.230619 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38261
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:38261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:08.314594 10246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:08.314817 10246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:08.314841 10246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:08.316130 10246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:08.316177 10246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:08.316192 10246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:08.316205 10246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:08.316218 10246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:08.318001 10246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:38261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:08.318308 10246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:08.318590 10246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:08.323814 10265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.323788 10262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.323910 10261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:08.323961 10246 server_base.cc:1034] running on GCE node
I20260811 07:07:08.324374 10246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:08.324679 10246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:08.325825 10246 hybrid_clock.cc:648] HybridClock initialized: now 1786432028325811 us; error 32 us; skew 500 ppm
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:08.328449 10246 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:08.328918 10246 webserver.cc:460] Webserver started at http://127.3.236.190:44727/ using document root <none> and password file <none>
I20260811 07:07:08.329164 10246 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:08.329210 10246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:08.329353 10246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:08.330411 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "37989484168e4a3ab63abc14be53ad69"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "2af1b51a491bdc9a73d87070de399643"
server_key_iv: "e1ef4f4da33fe74389549a9b29086f40"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.330780 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "37989484168e4a3ab63abc14be53ad69"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "2af1b51a491bdc9a73d87070de399643"
server_key_iv: "e1ef4f4da33fe74389549a9b29086f40"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.332125 10246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:07:08.332895 10280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.333112 10246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:08.333207 10246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "37989484168e4a3ab63abc14be53ad69"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "2af1b51a491bdc9a73d87070de399643"
server_key_iv: "e1ef4f4da33fe74389549a9b29086f40"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.333262 10246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:08.349968 10246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:08.351912 10246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:08.352034 10246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:08.356439 10246 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:38261
I20260811 07:07:08.356702 10358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:38261 every 8 connection(s)
I20260811 07:07:08.356884 10246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:08.357625 10359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:08.359949 10359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69: Bootstrap starting.
I20260811 07:07:08.360683 10359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:08.360994 10359 log.cc:824] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:08.361721 10359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69: No bootstrap required, opened a new log
I20260811 07:07:08.363337 10359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } }
I20260811 07:07:08.363502 10359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:08.363549 10359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37989484168e4a3ab63abc14be53ad69, State: Initialized, Role: FOLLOWER
I20260811 07:07:08.363729 10359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } }
I20260811 07:07:08.363817 10359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:08.363857 10359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:08.363909 10359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:08.364809 10359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } }
I20260811 07:07:08.364931 10359 leader_election.cc:302] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37989484168e4a3ab63abc14be53ad69; no voters:
I20260811 07:07:08.365132 10359 leader_election.cc:288] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:08.365182 10366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:08.365401 10359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:08.365427 10366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [term 1 LEADER]: Becoming Leader. State: Replica: 37989484168e4a3ab63abc14be53ad69, State: Running, Role: LEADER
I20260811 07:07:08.365535 10366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } }
I20260811 07:07:08.365973 10367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37989484168e4a3ab63abc14be53ad69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } } }
I20260811 07:07:08.366006 10368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37989484168e4a3ab63abc14be53ad69. Latest consensus state: current_term: 1 leader_uuid: "37989484168e4a3ab63abc14be53ad69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37989484168e4a3ab63abc14be53ad69" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38261 } } }
I20260811 07:07:08.366071 10367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:08.366101 10368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:08.366333 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10246
I20260811 07:07:08.366460 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:08.366652 10379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:08.366684 4018 external_mini_cluster.cc:1428] Setting key 00db9f306331f6b059f25a5af413bc69
I20260811 07:07:08.367228 10379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:08.368943 10379 catalog_manager.cc:1347] Generated new cluster ID: 8b45a6594adb4b2d88ad2215945421f3
I20260811 07:07:08.369001 10379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:08.377326 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.367363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.3.236.190:38261)
0811 07:07:08.367552 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.367562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.367597 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:07:08.367841 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.367847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.367923 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.368074 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.368914 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.369888 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.370671 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.371125 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.371256 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.373678 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.373682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.373683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.373703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.375711 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.376088 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.376092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.376093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.376193 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.376429 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.376430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.376431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.376682 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.376848 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.376958 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.377121 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":112,"threads_started":1}
I20260811 07:07:08.378625 10379 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:08.379103 10379 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:08.383956 10379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37989484168e4a3ab63abc14be53ad69: Generated new TSK 0
I20260811 07:07:08.384150 10379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:08.419219 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:08.527366 10395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:08.527595 10395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:08.527635 10395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:08.528892 10395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:08.528962 10395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:08.529007 10395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:08.530581 10395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:08.530843 10395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:08.531087 10395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:08.536278 10407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.536259 10417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.536369 10410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:08.536550 10395 server_base.cc:1034] running on GCE node
I20260811 07:07:08.536783 10395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:08.537051 10395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:08.538225 10395 hybrid_clock.cc:648] HybridClock initialized: now 1786432028538217 us; error 42 us; skew 500 ppm
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:08.540820 10395 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:08.541246 10395 webserver.cc:460] Webserver started at http://127.3.236.129:36761/ using document root <none> and password file <none>
I20260811 07:07:08.541479 10395 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:08.541558 10395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:08.541725 10395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:08.542802 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "a130fce00a41436385aeb6bebf3703aa"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "d95814901f44ad2ce7c3ed482fbdc939"
server_key_iv: "b5c93172fb321cabcd392c26cc7f02f5"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.543196 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "a130fce00a41436385aeb6bebf3703aa"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "d95814901f44ad2ce7c3ed482fbdc939"
server_key_iv: "b5c93172fb321cabcd392c26cc7f02f5"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.544647 10395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:08.545483 10427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.545701 10395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:08.545807 10395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "a130fce00a41436385aeb6bebf3703aa"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "d95814901f44ad2ce7c3ed482fbdc939"
server_key_iv: "b5c93172fb321cabcd392c26cc7f02f5"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.545892 10395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:08.555076 10395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:08.557534 10395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:08.557714 10395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:08.557972 10395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:08.558300 10395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:08.558358 10395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.558400 10395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:08.558430 10395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.564589 10395 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:40119
I20260811 07:07:08.564679 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:40119 every 8 connection(s)
I20260811 07:07:08.565039 10395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:08.566111 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10395
I20260811 07:07:08.566255 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:08.566489 4018 external_mini_cluster.cc:1428] Setting key f3723eba356e8706cde9c7620597e313
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:07:08.576867 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.565959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:33517 (local address 127.3.236.190:38261)
0811 07:07:08.566070 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.566073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.566374 (+ 301us) server_negotiation.cc:407] Connection header received
0811 07:07:08.567759 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.567760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.567770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.567811 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.568840 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.569433 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.570613 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.570847 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.570942 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.573732 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.573734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.573734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.573744 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.575269 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.575776 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.575777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.575777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.575837 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.576112 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.576113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.576114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.576232 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.576353 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.576678 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.576771 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:08.577059 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.565956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38261 (local address 127.3.236.129:33517)
0811 07:07:08.566194 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:08.566262 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:08.567649 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:08.567892 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:08.567896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:08.568423 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:08.568726 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.568733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.569562 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.569563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.570495 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.570497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.570957 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.570958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.571100 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.571712 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:08.571721 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:08.573572 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:08.575408 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.575416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.575419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.575683 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.575915 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.575916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.575917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.576028 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.576348 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:08.576356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:08.576518 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 07:07:08.576642 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 07:07:08.576813 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":114,"threads_started":1}
I20260811 07:07:08.577607 10603 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38261
I20260811 07:07:08.577724 10603 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:08.577912 10603 heartbeater.cc:511] Master 127.3.236.190:38261 requested a full tablet report, sending...
I20260811 07:07:08.578473 10304 ts_manager.cc:194] Registered new tserver with Master: a130fce00a41436385aeb6bebf3703aa (127.3.236.129:40119)
I20260811 07:07:08.579173 10304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:33517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:08.596382 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:08.681785 10610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:08.681996 10610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:08.682020 10610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:08.683110 10610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:08.683148 10610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:08.683174 10610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:08.684628 10610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:08.684808 10610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:08.684983 10610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:07:08.690284 10610 server_base.cc:1034] running on GCE node
W20260811 07:07:08.690212 10623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.690222 10620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.690222 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:08.690830 10610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:08.691071 10610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:08.692286 10610 hybrid_clock.cc:648] HybridClock initialized: now 1786432028692227 us; error 40 us; skew 500 ppm
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:08.694793 10610 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:08.695250 10610 webserver.cc:460] Webserver started at http://127.3.236.130:43263/ using document root <none> and password file <none>
I20260811 07:07:08.695482 10610 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:08.695554 10610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:08.695712 10610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:08.696869 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "342edde31d7741799ab38df35a40c83f"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "1e41816478b82b6dfbea1b5031ec0019"
server_key_iv: "6c8f825f398fd4419a02ca8b7e842db9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.697271 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "342edde31d7741799ab38df35a40c83f"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "1e41816478b82b6dfbea1b5031ec0019"
server_key_iv: "6c8f825f398fd4419a02ca8b7e842db9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.698642 10610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:08.699380 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.699554 10610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:08.699659 10610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "342edde31d7741799ab38df35a40c83f"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "1e41816478b82b6dfbea1b5031ec0019"
server_key_iv: "6c8f825f398fd4419a02ca8b7e842db9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.699739 10610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:08.712972 10610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:08.714922 10610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:08.715106 10610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:08.715505 10610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:08.715888 10610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:08.715950 10610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.716001 10610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:08.716051 10610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.722416 10610 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:45807
I20260811 07:07:08.722453 10822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:45807 every 8 connection(s)
I20260811 07:07:08.722792 10610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:08.731916 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.723555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:56825 (local address 127.3.236.190:38261)
0811 07:07:08.723648 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.723650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.723825 (+ 175us) server_negotiation.cc:407] Connection header received
0811 07:07:08.724847 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.724848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.724858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.724911 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.725492 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.726006 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.726934 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.727157 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.727209 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.729540 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.729542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.729542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.729548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.730717 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.731097 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.731098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.731098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.731150 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.731364 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.731364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.731365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.731465 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.731519 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.731748 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.731817 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:07:08.732158 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.723521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38261 (local address 127.3.236.130:56825)
0811 07:07:08.723700 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:08.723735 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:08.724756 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:08.724967 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:08.724971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:08.725228 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:08.725419 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.725423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.726086 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.726087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.726863 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.726864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.727236 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.727237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.727382 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.728014 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:08.728022 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:08.729426 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:08.730785 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.730792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.730793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.731029 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.731208 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.731209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.731209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.731309 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.731523 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:08.731528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:08.731639 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 07:07:08.731721 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:07:08.731867 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260811 07:07:08.732245 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10610
I20260811 07:07:08.732383 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:08.732651 4018 external_mini_cluster.cc:1428] Setting key 346bab4e52920147d1c0317a1bc62a33
I20260811 07:07:08.732879 10826 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38261
I20260811 07:07:08.732999 10826 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:08.733196 10826 heartbeater.cc:511] Master 127.3.236.190:38261 requested a full tablet report, sending...
I20260811 07:07:08.733611 10304 ts_manager.cc:194] Registered new tserver with Master: 342edde31d7741799ab38df35a40c83f (127.3.236.130:45807)
I20260811 07:07:08.734103 10304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:56825
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:08.759989 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:08.843626 10834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:08.843863 10834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:08.843910 10834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:08.845139 10834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:08.845207 10834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:08.845252 10834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:08.846797 10834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:08.847083 10834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:08.847328 10834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:08.852639 10847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.852648 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:08.852656 10846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:08.852833 10834 server_base.cc:1034] running on GCE node
I20260811 07:07:08.853179 10834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:08.853394 10834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:08.854575 10834 hybrid_clock.cc:648] HybridClock initialized: now 1786432028854557 us; error 41 us; skew 500 ppm
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:08.857355 10834 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:08.857792 10834 webserver.cc:460] Webserver started at http://127.3.236.131:43759/ using document root <none> and password file <none>
I20260811 07:07:08.858042 10834 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:08.858120 10834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:08.858278 10834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:08.859328 10834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "3c95a585b2e749fab04f4957dea7dcf4"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "8f5be3ab8cc3c0263045eed8ca55cb20"
server_key_iv: "f5f4e45792e0fe4061dd1a02ecbfdcc9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.859696 10834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "3c95a585b2e749fab04f4957dea7dcf4"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "8f5be3ab8cc3c0263045eed8ca55cb20"
server_key_iv: "f5f4e45792e0fe4061dd1a02ecbfdcc9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.861146 10834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:08.861919 10861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.862111 10834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:08.862223 10834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "3c95a585b2e749fab04f4957dea7dcf4"
format_stamp: "Formatted at 2026-08-11 07:07:08 on dist-test-slave-2d19"
server_key: "8f5be3ab8cc3c0263045eed8ca55cb20"
server_key_iv: "f5f4e45792e0fe4061dd1a02ecbfdcc9"
server_key_version: "encryptionkey@0"
I20260811 07:07:08.862300 10834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:08.873888 10834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:08.875875 10834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:08.876040 10834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:08.876348 10834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:08.876662 10834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:08.876718 10834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.876758 10834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:08.876785 10834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:08.882936 10834 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35987
I20260811 07:07:08.882992 11048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35987 every 8 connection(s)
I20260811 07:07:08.883332 10834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:08.885607 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10834
I20260811 07:07:08.885689 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:08.885874 4018 external_mini_cluster.cc:1428] Setting key a571c981a6e9ea0c1a6fc4f2e07fe10a
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:08.893007 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.884149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:47317 (local address 127.3.236.190:38261)
0811 07:07:08.884191 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.884193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.884448 (+ 255us) server_negotiation.cc:407] Connection header received
0811 07:07:08.885469 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.885471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.885485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.885534 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.886365 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.886989 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.887995 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.888210 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.888304 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.890351 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.890352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.890353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.890361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.891597 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.892003 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.892004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.892004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.892064 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.892366 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.892367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.892367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.892484 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.892555 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.892835 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.892917 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 07:07:08.893188 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.884132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38261 (local address 127.3.236.131:47317)
0811 07:07:08.884312 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:08.884366 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:08.885331 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:08.885617 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:08.885621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:08.885990 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:08.886256 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.886262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.887084 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.887085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.887888 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:08.887890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.888333 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.888334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:08.888471 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.888903 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:08.888911 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:08.890255 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:08.891680 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.891687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.891690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.891922 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.892142 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:08.892143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:08.892144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:08.892298 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:08.892557 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:08.892565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:08.892697 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 07:07:08.892796 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:07:08.892948 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260811 07:07:08.893680 11049 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38261
I20260811 07:07:08.893777 11049 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:08.893919 11049 heartbeater.cc:511] Master 127.3.236.190:38261 requested a full tablet report, sending...
I20260811 07:07:08.894277 10304 ts_manager.cc:194] Registered new tserver with Master: 3c95a585b2e749fab04f4957dea7dcf4 (127.3.236.131:35987)
I20260811 07:07:08.894654 10304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:47317
I20260811 07:07:08.897787 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:07:08.921643 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.913940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.3.236.129:40119)
0811 07:07:08.914134 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.914139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.914151 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:08.914179 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.914181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.914195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.914265 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.914886 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.915638 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.916501 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.916845 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.916913 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.919110 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.919118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.919118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.919125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.920590 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.920983 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.920985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.920986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.921037 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.921207 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.921208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.921208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.921335 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.921444 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.921504 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.921565 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
Aug 11 07:07:08 dist-test-slave-2d19 krb5kdc[10223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:08.929615 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:08.923040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.3.236.190:38261)
0811 07:07:08.923097 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:08.923099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:08.923110 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:08.923183 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:08.923184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:08.923194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:08.923267 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:08.923743 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.924234 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.924966 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:08.925163 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:08.925240 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:08.927224 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:08.927226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:08.927226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:08.927233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:08.928452 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.928780 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.928781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.928781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.928842 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:08.929055 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:08.929055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:08.929056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:08.929208 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:08.929315 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:08.929433 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 07:07:08.929509 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 07:07:08.929832 10304 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41818
I20260811 07:07:08.930202 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10395
I20260811 07:07:08.943897 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10610
I20260811 07:07:08.958163 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10834
I20260811 07:07:08.976907 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10246
2026-08-11T07:07:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2147 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:09 dist-test-slave-2d19 krb5kdc[11064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:09 dist-test-slave-2d19 krb5kdc[11064](info): set up 2 sockets
Aug 11 07:07:09 dist-test-slave-2d19 krb5kdc[11064](info): commencing operation
krb5kdc: starting...
W20260811 07:07:10.267910 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:10.392004 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46845
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:10.475436 11084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:10.475693 11084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:10.475732 11084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:10.476918 11084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:10.476989 11084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:10.477013 11084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:10.477035 11084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:10.477066 11084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:10.478601 11084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:10.478900 11084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:10.479136 11084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:10.484058 11099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:10.484067 11094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.484158 11084 server_base.cc:1034] running on GCE node
W20260811 07:07:10.484076 11108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.484622 11084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:10.484920 11084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:10.486101 11084 hybrid_clock.cc:648] HybridClock initialized: now 1786432030486080 us; error 34 us; skew 500 ppm
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:10.488673 11084 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:10.489058 11084 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:10.489130 11084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:10.489269 11084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:10.490335 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "7814dc7b52744263abaff5db72fef30f"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "af5aa4e98c275a739fe1d55106be077b"
server_key_iv: "60ce655e390e950f37b2b476bd1e5b32"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.490717 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "7814dc7b52744263abaff5db72fef30f"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "af5aa4e98c275a739fe1d55106be077b"
server_key_iv: "60ce655e390e950f37b2b476bd1e5b32"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.492045 11084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:10.492862 11115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.493039 11084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.493139 11084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "7814dc7b52744263abaff5db72fef30f"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "af5aa4e98c275a739fe1d55106be077b"
server_key_iv: "60ce655e390e950f37b2b476bd1e5b32"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.493227 11084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:10.511775 11084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:10.513715 11084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:10.513882 11084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:10.518222 11084 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:46845
I20260811 07:07:10.518252 11206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:46845 every 8 connection(s)
I20260811 07:07:10.518558 11084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:10.519661 11207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:10.522058 11207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: Bootstrap starting.
I20260811 07:07:10.522636 11207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:10.522887 11207 log.cc:824] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:10.523571 11207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: No bootstrap required, opened a new log
I20260811 07:07:10.525172 11207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } }
I20260811 07:07:10.525314 11207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:10.525338 11207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7814dc7b52744263abaff5db72fef30f, State: Initialized, Role: FOLLOWER
I20260811 07:07:10.525447 11207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } }
I20260811 07:07:10.525539 11207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:10.525583 11207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:10.525637 11207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:10.526449 11207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } }
I20260811 07:07:10.526577 11207 leader_election.cc:302] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7814dc7b52744263abaff5db72fef30f; no voters:
I20260811 07:07:10.526765 11207 leader_election.cc:288] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:10.526825 11210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:10.527031 11207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:10.527040 11210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [term 1 LEADER]: Becoming Leader. State: Replica: 7814dc7b52744263abaff5db72fef30f, State: Running, Role: LEADER
I20260811 07:07:10.527271 11210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } }
I20260811 07:07:10.527292 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 11084
I20260811 07:07:10.527415 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:10.527626 4018 external_mini_cluster.cc:1428] Setting key 85708ec3a60d7059b5cbff7b2c942d51
I20260811 07:07:10.527684 11214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7814dc7b52744263abaff5db72fef30f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } } }
I20260811 07:07:10.527740 11215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7814dc7b52744263abaff5db72fef30f. Latest consensus state: current_term: 1 leader_uuid: "7814dc7b52744263abaff5db72fef30f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7814dc7b52744263abaff5db72fef30f" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46845 } } }
I20260811 07:07:10.527770 11214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:10.527796 11215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:10.528216 11224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:10.528955 11224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:10.530881 11224 catalog_manager.cc:1347] Generated new cluster ID: 4ef4038fe8af4407bf901d07643c6c64
I20260811 07:07:10.530947 11224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:10.540195 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:10.528139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.3.236.190:46845)
0811 07:07:10.528344 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:10.528353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:10.528598 (+ 245us) server_negotiation.cc:407] Connection header received
0811 07:07:10.528876 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:10.528883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:10.528942 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:10.529854 (+ 912us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:10.529964 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.531373 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.532545 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.533216 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.533327 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:10.536306 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:10.536311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:10.536311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:10.536324 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:10.538449 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.538865 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.538868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.538869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.538988 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.539230 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.539231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.539231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.539556 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:10.539720 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:10.539825 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 07:07:10.539966 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260811 07:07:10.545696 11224 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:10.546253 11224 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:10.550772 11224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: Generated new TSK 0
I20260811 07:07:10.550946 11224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:10.580857 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:10.664423 11265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:10.664650 11265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:10.664688 11265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:10.665834 11265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:10.665900 11265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:10.665953 11265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:10.667488 11265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:10.667752 11265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:10.667991 11265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:10.673269 11276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.673498 11265 server_base.cc:1034] running on GCE node
W20260811 07:07:10.673288 11277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:10.673247 11280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.673808 11265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:10.674134 11265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:10.675323 11265 hybrid_clock.cc:648] HybridClock initialized: now 1786432030675307 us; error 27 us; skew 500 ppm
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:10.677942 11265 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:10.678360 11265 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:10.678442 11265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:10.678609 11265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:10.679665 11265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "ca48d46a9313460d85eb44e3a886e9c1"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "249254fb201ed735841908179b3b0099"
server_key_iv: "1674ab0d8ae061b9b452ff49eb7dcafd"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.680033 11265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "ca48d46a9313460d85eb44e3a886e9c1"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "249254fb201ed735841908179b3b0099"
server_key_iv: "1674ab0d8ae061b9b452ff49eb7dcafd"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.681424 11265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:10.682194 11288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.682379 11265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:10.682482 11265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "ca48d46a9313460d85eb44e3a886e9c1"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "249254fb201ed735841908179b3b0099"
server_key_iv: "1674ab0d8ae061b9b452ff49eb7dcafd"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.682574 11265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:10.693795 11265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:10.695536 11265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:10.695710 11265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:10.695974 11265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:10.696413 11265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:10.696476 11265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.696523 11265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:10.696553 11265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.702569 11265 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:38241
I20260811 07:07:10.702625 11493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:38241 every 8 connection(s)
I20260811 07:07:10.702899 11265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:10.706408 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 11265
I20260811 07:07:10.706521 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:10.706715 4018 external_mini_cluster.cc:1428] Setting key 0eb87ed10a34fd1fae33223db1112ab3
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:10.713389 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:10.703652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:45651 (local address 127.3.236.190:46845)
0811 07:07:10.703720 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:10.703722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:10.703948 (+ 226us) server_negotiation.cc:407] Connection header received
0811 07:07:10.704491 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:10.704492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:10.704505 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:10.704564 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:10.705468 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.705959 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.707247 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.707490 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.707544 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:10.710362 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:10.710364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:10.710365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:10.710376 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:10.711890 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.712358 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.712360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.712360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.712431 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.712724 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.712725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.712726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.712845 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:10.712923 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:10.713211 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 07:07:10.713297 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:10.713588 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:10.703636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46845 (local address 127.3.236.129:45651)
0811 07:07:10.703815 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:10.703856 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:10.704404 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:10.704618 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:10.704621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:10.704889 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:10.705388 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:10.705393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.706040 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.706041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:10.707155 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:10.707158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.707576 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.707577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:10.707725 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:10.708235 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:10.708246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:10.710257 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:10.712002 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:10.712011 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:10.712013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:10.712245 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:10.712509 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:10.712510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:10.712510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:10.712643 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:10.712922 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:10.712930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:10.713069 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 07:07:10.713175 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 07:07:10.713335 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260811 07:07:10.714320 11496 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46845
I20260811 07:07:10.714540 11496 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:10.714699 11496 heartbeater.cc:511] Master 127.3.236.190:46845 requested a full tablet report, sending...
I20260811 07:07:10.715364 11149 ts_manager.cc:194] Registered new tserver with Master: ca48d46a9313460d85eb44e3a886e9c1 (127.3.236.129:38241)
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:07:10.716378 11149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:45651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:10.736394 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:10.819278 11509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:10.819476 11509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:10.819500 11509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:10.820715 11509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:10.820757 11509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:10.820783 11509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:10.822295 11509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:10.822544 11509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:10.822793 11509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:10.827831 11519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:10.827888 11520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:10.827831 11522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.828071 11509 server_base.cc:1034] running on GCE node
I20260811 07:07:10.828310 11509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:10.828559 11509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:10.829734 11509 hybrid_clock.cc:648] HybridClock initialized: now 1786432030829715 us; error 30 us; skew 500 ppm
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:10.832334 11509 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:10.832684 11509 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:10.832759 11509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:10.832921 11509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:10.834055 11509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "130c47559c9044b6baab9af77886cf87"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "7302b851c57bd84f368e555aa586423e"
server_key_iv: "14aa97a43f60b07a568a127c3e76d01b"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.834434 11509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "130c47559c9044b6baab9af77886cf87"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "7302b851c57bd84f368e555aa586423e"
server_key_iv: "14aa97a43f60b07a568a127c3e76d01b"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.835779 11509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:10.836629 11534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.836802 11509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.836913 11509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "130c47559c9044b6baab9af77886cf87"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "7302b851c57bd84f368e555aa586423e"
server_key_iv: "14aa97a43f60b07a568a127c3e76d01b"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.837018 11509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:10.844388 11509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:10.846156 11509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:10.846318 11509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:10.846561 11509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:10.846877 11509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:10.846933 11509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.846982 11509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:10.847029 11509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.853070 11509 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:35511
I20260811 07:07:10.853103 11717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:35511 every 8 connection(s)
I20260811 07:07:10.853377 11509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:10.862210 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 11509
I20260811 07:07:10.862315 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:10.862531 4018 external_mini_cluster.cc:1428] Setting key 5928927bef51f2651ca47f708fac6814
I20260811 07:07:10.862682 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:10.854149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:43763 (local address 127.3.236.190:46845)
0811 07:07:10.854219 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:10.854221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:10.854439 (+ 218us) server_negotiation.cc:407] Connection header received
0811 07:07:10.855020 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:10.855021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:10.855032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:10.855093 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:10.856063 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.856632 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.857566 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.857745 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.857794 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:10.860059 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:10.860060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:10.860061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:10.860067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:10.861336 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.861702 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.861703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.861703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.861754 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:10.861962 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:10.861962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:10.861963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:10.862081 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:10.862134 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:10.862485 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 07:07:10.862554 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:10.862926 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:10.854123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46845 (local address 127.3.236.130:43763)
0811 07:07:10.854307 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:10.854348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:10.854896 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:10.855172 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:10.855175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:10.855427 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:10.855971 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:10.855976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.856712 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.856713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:10.857495 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:10.857497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:10.857816 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:10.857817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:10.857915 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:10.858445 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:10.858453 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:10.859934 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:10.861407 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:10.861415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:10.861416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:10.861638 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:10.861813 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:10.861814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:10.861814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:10.861909 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:10.862154 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:10.862162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:10.862335 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 07:07:10.862467 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 07:07:10.862658 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260811 07:07:10.863487 11718 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46845
I20260811 07:07:10.863606 11718 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:10.863778 11718 heartbeater.cc:511] Master 127.3.236.190:46845 requested a full tablet report, sending...
I20260811 07:07:10.864185 11149 ts_manager.cc:194] Registered new tserver with Master: 130c47559c9044b6baab9af77886cf87 (127.3.236.130:35511)
I20260811 07:07:10.864586 11149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:43763
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:10.891114 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:10.975801 11724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:10.976050 11724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:10.976095 11724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:10.977722 11724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:10.977775 11724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:10.977821 11724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:10.979498 11724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:10.979768 11724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:10.980086 11724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:10.985075 11734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:10.985149 11736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.985247 11724 server_base.cc:1034] running on GCE node
W20260811 07:07:10.985154 11740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:10.985647 11724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:10.985909 11724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:10.987082 11724 hybrid_clock.cc:648] HybridClock initialized: now 1786432030987069 us; error 49 us; skew 500 ppm
Aug 11 07:07:10 dist-test-slave-2d19 krb5kdc[11064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:10.989686 11724 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:10.990003 11724 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:10.990049 11724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:10.990140 11724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:10.991180 11724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "12928960c5a84200befdc79e1e816b2a"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "78766fdee24d48385d9d0049ab3660e1"
server_key_iv: "ad1d3d0ba0853515dce470110374f0ba"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.991494 11724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "12928960c5a84200befdc79e1e816b2a"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "78766fdee24d48385d9d0049ab3660e1"
server_key_iv: "ad1d3d0ba0853515dce470110374f0ba"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.992944 11724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:10.993683 11748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:10.993861 11724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:10.993957 11724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "12928960c5a84200befdc79e1e816b2a"
format_stamp: "Formatted at 2026-08-11 07:07:10 on dist-test-slave-2d19"
server_key: "78766fdee24d48385d9d0049ab3660e1"
server_key_iv: "ad1d3d0ba0853515dce470110374f0ba"
server_key_version: "encryptionkey@0"
I20260811 07:07:10.994058 11724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:11.007436 11724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:11.009434 11724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:11.009598 11724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:11.009843 11724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:11.010187 11724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:11.010228 11724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.010253 11724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:11.010303 11724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.016603 11724 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:42795
I20260811 07:07:11.016647 11940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:42795 every 8 connection(s)
I20260811 07:07:11.016918 11724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:07:11 dist-test-slave-2d19 krb5kdc[11064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432030, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:11.026230 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.017616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:59021 (local address 127.3.236.190:46845)
0811 07:07:11.017689 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.017691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.017923 (+ 232us) server_negotiation.cc:407] Connection header received
0811 07:07:11.018418 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.018419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.018430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.018488 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:11.019462 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.019945 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.021023 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.021214 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.021269 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.023658 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:11.023659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:11.023660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:11.023667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:11.025069 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:11.025445 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:11.025445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:11.025446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:11.025495 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:11.025711 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:11.025711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:11.025711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:11.025807 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:11.025856 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.026099 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.026165 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:07:11.026306 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 11724
I20260811 07:07:11.026398 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:11.026525 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.017592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46845 (local address 127.3.236.131:59021)
0811 07:07:11.017789 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.017833 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.018320 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.018570 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.018574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.018845 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:11.019370 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.019375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.020031 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.020031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.020938 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.020941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.021320 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.021322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.021453 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.021975 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:11.021984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:11.023564 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:11.025150 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:11.025156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:11.025158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:11.025378 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:11.025554 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:11.025554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:11.025554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:11.025656 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:11.025861 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:11.025865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:11.025980 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.026063 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.026248 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260811 07:07:11.026645 4018 external_mini_cluster.cc:1428] Setting key 525c45f4c867621277b72a63811c4acb
I20260811 07:07:11.027047 11946 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46845
I20260811 07:07:11.027151 11946 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:11.027323 11946 heartbeater.cc:511] Master 127.3.236.190:46845 requested a full tablet report, sending...
I20260811 07:07:11.027720 11149 ts_manager.cc:194] Registered new tserver with Master: 12928960c5a84200befdc79e1e816b2a (127.3.236.131:42795)
I20260811 07:07:11.028198 11149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:59021
I20260811 07:07:11.038549 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:07:11.046355 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.039944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.3.236.190:46845)
0811 07:07:11.040035 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.040038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.040168 (+ 130us) server_negotiation.cc:407] Connection header received
0811 07:07:11.040248 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.040271 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.040282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.040345 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:11.041018 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.041512 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.042287 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.042493 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.042579 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.043803 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:11.043804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:11.043805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:11.043812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:11.045161 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:11.045509 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:11.045510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:11.045511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:11.045572 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:11.045777 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:11.045778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:11.045779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:11.045932 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:11.046048 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.046164 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.046247 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:11.047952 11149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:11.049685 11149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:11.056414 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:35511 (local address 127.0.0.1:36968)
0811 07:07:11.052564 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.052574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.052663 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.053020 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.053021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.053023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:11.053073 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.053075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.053968 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.053969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.055271 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.055274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.056072 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.056154 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056167 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.056282 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.056339 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260811 07:07:11.056430 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.3.236.130:35511)
0811 07:07:11.052498 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.052504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.052702 (+ 198us) server_negotiation.cc:407] Connection header received
0811 07:07:11.052823 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.052825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.052842 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.052922 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:11.053155 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.053864 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.055413 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.055959 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056068 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056131 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.056294 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.056336 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260811 07:07:11.056639 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:42795 (local address 127.0.0.1:37350)
0811 07:07:11.052386 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.052428 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.052496 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.052830 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.052833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.052835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:11.052904 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.052906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.054232 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.054233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.055465 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.055467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.056405 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.056470 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056472 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.056520 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.056555 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":30,"thread_start_us":70,"threads_started":1}
I20260811 07:07:11.056784 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.3.236.129:38241)
0811 07:07:11.052875 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.052879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.052888 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:11.052925 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.052927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.052943 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.053026 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:11.053234 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.054257 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.055340 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.056089 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056171 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056229 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.056633 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.056683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":88,"threads_started":1}
I20260811 07:07:11.056927 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.3.236.131:42795)
0811 07:07:11.052540 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.052545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.052557 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:11.052617 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.052619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.052637 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.052711 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:11.052994 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.054129 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.055536 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.056217 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056317 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.056798 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.056844 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":149,"threads_started":1}
I20260811 07:07:11.056977 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.052282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38241 (local address 127.0.0.1:55788)
0811 07:07:11.052590 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.052599 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.052637 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.053111 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.053112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.053114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:11.053148 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.053149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.054364 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.054365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.055256 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.055259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.056296 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.056297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.056355 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.056359 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.056408 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.056449 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":3,"thread_start_us":37,"threads_started":1}
I20260811 07:07:11.057614 11616 tablet_service.cc:1467] Processing CreateTablet for tablet 05b9c60726124115b871dde7e811dfbf (DEFAULT_TABLE table=test-table [id=ec06d7e098d34cd38ee44b9d01ba5990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:11.057680 11375 tablet_service.cc:1467] Processing CreateTablet for tablet 05b9c60726124115b871dde7e811dfbf (DEFAULT_TABLE table=test-table [id=ec06d7e098d34cd38ee44b9d01ba5990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:11.057680 11826 tablet_service.cc:1467] Processing CreateTablet for tablet 05b9c60726124115b871dde7e811dfbf (DEFAULT_TABLE table=test-table [id=ec06d7e098d34cd38ee44b9d01ba5990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:11.057911 11616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b9c60726124115b871dde7e811dfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:11.058029 11826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b9c60726124115b871dde7e811dfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:11.058029 11375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b9c60726124115b871dde7e811dfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:11.060858 11971 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Bootstrap starting.
I20260811 07:07:11.061534 11975 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Bootstrap starting.
I20260811 07:07:11.061570 11971 tablet_bootstrap.cc:654] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:11.062222 11971 log.cc:824] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:11.062328 11975 tablet_bootstrap.cc:654] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:11.062927 11975 log.cc:824] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:11.063093 11971 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: No bootstrap required, opened a new log
I20260811 07:07:11.063177 11971 ts_tablet_manager.cc:1397] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:07:11.063301 11977 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Bootstrap starting.
I20260811 07:07:11.063958 11977 tablet_bootstrap.cc:654] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:11.064566 11975 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: No bootstrap required, opened a new log
I20260811 07:07:11.064591 11977 log.cc:824] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:11.064652 11975 ts_tablet_manager.cc:1397] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:07:11.065290 11971 raft_consensus.cc:348] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.065428 11971 raft_consensus.cc:374] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:11.065448 11971 raft_consensus.cc:729] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130c47559c9044b6baab9af77886cf87, State: Initialized, Role: FOLLOWER
I20260811 07:07:11.065569 11971 consensus_queue.cc:260] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.065867 11971 ts_tablet_manager.cc:1428] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:07:11.065900 11718 heartbeater.cc:503] Master 127.3.236.190:46845 was elected leader, sending a full tablet report...
I20260811 07:07:11.066552 11977 tablet_bootstrap.cc:492] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: No bootstrap required, opened a new log
I20260811 07:07:11.066639 11977 ts_tablet_manager.cc:1397] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:07:11.066670 11975 raft_consensus.cc:348] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.066857 11975 raft_consensus.cc:374] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:11.066902 11975 raft_consensus.cc:729] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca48d46a9313460d85eb44e3a886e9c1, State: Initialized, Role: FOLLOWER
I20260811 07:07:11.067015 11975 consensus_queue.cc:260] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.067296 11975 ts_tablet_manager.cc:1428] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:07:11.067328 11496 heartbeater.cc:503] Master 127.3.236.190:46845 was elected leader, sending a full tablet report...
I20260811 07:07:11.068885 11977 raft_consensus.cc:348] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.069080 11977 raft_consensus.cc:374] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:11.069132 11977 raft_consensus.cc:729] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12928960c5a84200befdc79e1e816b2a, State: Initialized, Role: FOLLOWER
I20260811 07:07:11.069249 11977 consensus_queue.cc:260] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.069542 11977 ts_tablet_manager.cc:1428] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 07:07:11.069623 11946 heartbeater.cc:503] Master 127.3.236.190:46845 was elected leader, sending a full tablet report...
W20260811 07:07:11.103886 11719 tablet.cc:2369] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:07:11.203673 11501 tablet.cc:2369] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:07:11.267490 11949 tablet.cc:2369] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:07:11.270089 11985 raft_consensus.cc:482] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:11.270208 11985 raft_consensus.cc:504] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.270700 11985 leader_election.cc:288] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12928960c5a84200befdc79e1e816b2a (127.3.236.131:42795), ca48d46a9313460d85eb44e3a886e9c1 (127.3.236.129:38241)
I20260811 07:07:11.273618 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.270881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:42795 (local address 127.3.236.130:38023)
0811 07:07:11.270977 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.270990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.271040 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.271227 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.271228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.271230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:11.271308 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.271310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.271959 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.271960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.272791 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.272793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.273354 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.273355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.273431 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.273434 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.273499 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.273540 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 07:07:11.273609 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.270962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:38023 (local address 127.3.236.131:42795)
0811 07:07:11.271082 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.271085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.271119 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:07:11.271144 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.271145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.271153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.271221 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:11.271386 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.271858 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.272892 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.273263 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.273345 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.273355 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.273506 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.273541 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:07:11.274034 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.270891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38241 (local address 127.3.236.130:34309)
0811 07:07:11.271149 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.271159 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.271251 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.271401 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.271402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.271404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:11.271485 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.271487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.272178 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.272179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.273079 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:11.273081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.273727 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.273729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:11.273809 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.273813 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.273905 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.273950 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":33,"thread_start_us":144,"threads_started":1}
I20260811 07:07:11.274046 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.270995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:34309 (local address 127.3.236.129:38241)
0811 07:07:11.271053 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.271055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.271236 (+ 181us) server_negotiation.cc:407] Connection header received
0811 07:07:11.271316 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.271317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.271328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.271397 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:11.271561 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.272080 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.273180 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.273600 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.273691 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.273699 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.273938 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.273981 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:07:11.274013 11859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b9c60726124115b871dde7e811dfbf" candidate_uuid: "130c47559c9044b6baab9af77886cf87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12928960c5a84200befdc79e1e816b2a" is_pre_election: true
I20260811 07:07:11.274215 11859 raft_consensus.cc:2454] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130c47559c9044b6baab9af77886cf87 in term 0.
I20260811 07:07:11.274451 11536 leader_election.cc:302] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12928960c5a84200befdc79e1e816b2a, 130c47559c9044b6baab9af77886cf87; no voters:
I20260811 07:07:11.274427 11413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b9c60726124115b871dde7e811dfbf" candidate_uuid: "130c47559c9044b6baab9af77886cf87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca48d46a9313460d85eb44e3a886e9c1" is_pre_election: true
I20260811 07:07:11.274610 11413 raft_consensus.cc:2454] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130c47559c9044b6baab9af77886cf87 in term 0.
I20260811 07:07:11.274616 11985 raft_consensus.cc:2781] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:11.274677 11985 raft_consensus.cc:482] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:11.274713 11985 raft_consensus.cc:3037] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:11.275439 11985 raft_consensus.cc:504] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.275575 11985 leader_election.cc:288] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [CANDIDATE]: Term 1 election: Requested vote from peers 12928960c5a84200befdc79e1e816b2a (127.3.236.131:42795), ca48d46a9313460d85eb44e3a886e9c1 (127.3.236.129:38241)
I20260811 07:07:11.275712 11859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b9c60726124115b871dde7e811dfbf" candidate_uuid: "130c47559c9044b6baab9af77886cf87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12928960c5a84200befdc79e1e816b2a"
I20260811 07:07:11.275754 11413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b9c60726124115b871dde7e811dfbf" candidate_uuid: "130c47559c9044b6baab9af77886cf87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca48d46a9313460d85eb44e3a886e9c1"
I20260811 07:07:11.275802 11859 raft_consensus.cc:3037] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:11.275844 11413 raft_consensus.cc:3037] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:11.276614 11859 raft_consensus.cc:2454] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130c47559c9044b6baab9af77886cf87 in term 1.
I20260811 07:07:11.276692 11413 raft_consensus.cc:2454] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130c47559c9044b6baab9af77886cf87 in term 1.
I20260811 07:07:11.276774 11536 leader_election.cc:302] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12928960c5a84200befdc79e1e816b2a, 130c47559c9044b6baab9af77886cf87; no voters:
I20260811 07:07:11.276880 11985 raft_consensus.cc:2781] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:11.277087 11985 raft_consensus.cc:686] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 1 LEADER]: Becoming Leader. State: Replica: 130c47559c9044b6baab9af77886cf87, State: Running, Role: LEADER
I20260811 07:07:11.277190 11985 consensus_queue.cc:237] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } }
I20260811 07:07:11.278124 11148 catalog_manager.cc:5495] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130c47559c9044b6baab9af77886cf87 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "130c47559c9044b6baab9af77886cf87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130c47559c9044b6baab9af77886cf87" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:11.339285 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.337000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.3.236.130:35511)
0811 07:07:11.337045 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.337047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.337091 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:07:11.337127 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.337129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.337147 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.337203 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:11.337351 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.337821 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.338528 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:11.338731 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:11.338782 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:11.338902 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:11.339030 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:11.339089 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.339190 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.339214 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:07:11.341295 11413 raft_consensus.cc:1261] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 1 FOLLOWER]: Refusing update from remote peer 130c47559c9044b6baab9af77886cf87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:11.341298 11859 raft_consensus.cc:1261] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 1 FOLLOWER]: Refusing update from remote peer 130c47559c9044b6baab9af77886cf87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:11.341681 11985 consensus_queue.cc:1035] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca48d46a9313460d85eb44e3a886e9c1" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:11.341822 11993 consensus_queue.cc:1035] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12928960c5a84200befdc79e1e816b2a" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:11.345052 12001 mvcc.cc:205] Tried to move back new op lower bound from 7317225600370749440 to 7317225600111804416. Current Snapshot: MvccSnapshot[applied={T|T < 7317225600370749440}]
I20260811 07:07:11.346566 12002 mvcc.cc:205] Tried to move back new op lower bound from 7317225600370749440 to 7317225600111804416. Current Snapshot: MvccSnapshot[applied={T|T < 7317225600370749440 or (T in {7317225600370749440})}]
I20260811 07:07:11.346771 11148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:11.346879 11148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:11.347752 11148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7814dc7b52744263abaff5db72fef30f: Sending DeleteTablet for 3 replicas of tablet 05b9c60726124115b871dde7e811dfbf
I20260811 07:07:11.348111 11826 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b9c60726124115b871dde7e811dfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:11 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:37350
I20260811 07:07:11.348167 11616 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b9c60726124115b871dde7e811dfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:11 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:36968
I20260811 07:07:11.348409 12011 tablet_replica.cc:330] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: stopping tablet replica
I20260811 07:07:11.348436 11375 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b9c60726124115b871dde7e811dfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:11 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55788
I20260811 07:07:11.348511 12011 raft_consensus.cc:2229] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:11.348634 12011 raft_consensus.cc:2258] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:11.348677 12014 tablet_replica.cc:330] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1: stopping tablet replica
I20260811 07:07:11.348698 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 11265
I20260811 07:07:11.348793 12014 raft_consensus.cc:2229] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:11.348905 12014 raft_consensus.cc:2258] T 05b9c60726124115b871dde7e811dfbf P ca48d46a9313460d85eb44e3a886e9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:11.355469 12012 tablet_replica.cc:330] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: stopping tablet replica
I20260811 07:07:11.355600 12012 raft_consensus.cc:2229] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:11.355703 12012 raft_consensus.cc:2258] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:11.360368 12012 ts_tablet_manager.cc:1905] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:11.362484 12011 ts_tablet_manager.cc:1905] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:11.362622 12012 ts_tablet_manager.cc:1918] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:11.362756 12012 log.cc:1196] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/05b9c60726124115b871dde7e811dfbf
I20260811 07:07:11.363122 12012 ts_tablet_manager.cc:1939] T 05b9c60726124115b871dde7e811dfbf P 130c47559c9044b6baab9af77886cf87: Deleting consensus metadata
I20260811 07:07:11.363639 11122 catalog_manager.cc:4841] TS 130c47559c9044b6baab9af77886cf87 (127.3.236.130:35511): tablet 05b9c60726124115b871dde7e811dfbf (table test-table [id=ec06d7e098d34cd38ee44b9d01ba5990]) successfully deleted
I20260811 07:07:11.364310 12011 ts_tablet_manager.cc:1918] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:11.364387 12011 log.cc:1196] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/05b9c60726124115b871dde7e811dfbf
I20260811 07:07:11.364670 12011 ts_tablet_manager.cc:1939] T 05b9c60726124115b871dde7e811dfbf P 12928960c5a84200befdc79e1e816b2a: Deleting consensus metadata
I20260811 07:07:11.365088 11118 catalog_manager.cc:4841] TS 12928960c5a84200befdc79e1e816b2a (127.3.236.131:42795): tablet 05b9c60726124115b871dde7e811dfbf (table test-table [id=ec06d7e098d34cd38ee44b9d01ba5990]) successfully deleted
W20260811 07:07:11.367678 11122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:38241 (error 108)
I20260811 07:07:11.367937 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 11509
I20260811 07:07:11.370057 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.368039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38241 (local address 127.0.0.1:55804)
0811 07:07:11.369970 (+ 1931us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.370014 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1905}
W20260811 07:07:11.371569 11122 catalog_manager.cc:4584] TS ca48d46a9313460d85eb44e3a886e9c1 (127.3.236.129:38241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05b9c60726124115b871dde7e811dfbf: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38241: connect: Connection refused (error 111)
I20260811 07:07:11.378839 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 11724
I20260811 07:07:11.391067 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 11084
2026-08-11T07:07:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2413 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T07:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:11Z Disabled control of system clock
I20260811 07:07:11.425698 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36385
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:36385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:07:11.508803 12022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:11.509006 12022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:11.509027 12022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:11.510074 12022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:11.510105 12022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:11.510118 12022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:11.510131 12022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:11.510143 12022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:11.511392 12022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:36385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:11.511579 12022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:11.511808 12022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:11.516829 12033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.516820 12031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.516820 12022 server_base.cc:1034] running on GCE node
W20260811 07:07:11.516852 12028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.517370 12022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:11.517706 12022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:11.518883 12022 hybrid_clock.cc:648] HybridClock initialized: now 1786432031518864 us; error 43 us; skew 500 ppm
I20260811 07:07:11.520208 12022 webserver.cc:460] Webserver started at http://127.3.236.190:45155/ using document root <none> and password file <none>
I20260811 07:07:11.520506 12022 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:11.520584 12022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:11.520743 12022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:11.521838 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "96fd1f84245c4df4a3a0a3b1c7154b35"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "0412d3e432c6a6b0b23a349b134ba9e0"
server_key_iv: "6a3953d16a9263a6647860671729b9df"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.522202 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "96fd1f84245c4df4a3a0a3b1c7154b35"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "0412d3e432c6a6b0b23a349b134ba9e0"
server_key_iv: "6a3953d16a9263a6647860671729b9df"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.523509 12022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:11.524335 12040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.524515 12022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:11.524619 12022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "96fd1f84245c4df4a3a0a3b1c7154b35"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "0412d3e432c6a6b0b23a349b134ba9e0"
server_key_iv: "6a3953d16a9263a6647860671729b9df"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.524705 12022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:11.528047 12022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:11.528360 12022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:11.528513 12022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:11.532564 12022 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:36385
I20260811 07:07:11.532601 12138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:36385 every 8 connection(s)
I20260811 07:07:11.532938 12022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:11.533653 12139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:11.536450 12139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: Bootstrap starting.
I20260811 07:07:11.537014 12139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:11.537256 12139 log.cc:824] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:11.537904 12139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: No bootstrap required, opened a new log
I20260811 07:07:11.539408 12139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } }
I20260811 07:07:11.539542 12139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:11.539565 12139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96fd1f84245c4df4a3a0a3b1c7154b35, State: Initialized, Role: FOLLOWER
I20260811 07:07:11.539707 12139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } }
I20260811 07:07:11.539789 12139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:11.539811 12139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:11.539840 12139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:11.540102 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 12022
I20260811 07:07:11.540213 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:11.540449 4018 external_mini_cluster.cc:1428] Setting key 2e38f9ce18ec8c9a98101eb1396183ca
I20260811 07:07:11.540606 12139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } }
I20260811 07:07:11.540743 12139 leader_election.cc:302] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96fd1f84245c4df4a3a0a3b1c7154b35; no voters:
I20260811 07:07:11.540930 12139 leader_election.cc:288] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:11.541016 12143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:11.541173 12143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [term 1 LEADER]: Becoming Leader. State: Replica: 96fd1f84245c4df4a3a0a3b1c7154b35, State: Running, Role: LEADER
I20260811 07:07:11.541206 12139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:11.541250 12143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } }
I20260811 07:07:11.541606 12143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } } }
I20260811 07:07:11.541699 12143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:11.541843 12143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96fd1f84245c4df4a3a0a3b1c7154b35. Latest consensus state: current_term: 1 leader_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96fd1f84245c4df4a3a0a3b1c7154b35" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36385 } } }
I20260811 07:07:11.541913 12143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:11.541931 12160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:11.542523 12160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:11.544413 12160 catalog_manager.cc:1347] Generated new cluster ID: 9a624dbc2dcf458b8064ecc46468af89
I20260811 07:07:11.544421 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.541055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.3.236.190:36385)
0811 07:07:11.541272 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.541281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.542300 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 07:07:11.542580 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.542589 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.542644 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.542773 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:11.543928 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:11.543932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:11.543932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:11.543937 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:11.543981 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:11.543995 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:11.544063 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.544149 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.544208 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260811 07:07:11.544488 12160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:11.597373 12160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:11.597568 12160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:11.602219 12160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: Generated new TSK 0
I20260811 07:07:11.602415 12160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:11.617031 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--builtin_ntp_servers=127.3.236.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:07:11.700930 12177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:11.701165 12177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:11.701205 12177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:11.702355 12177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:11.702426 12177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:11.702473 12177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:11.703742 12177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:11.703917 12177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:11.704157 12177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:11.709064 12184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.709069 12185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.709067 12192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.709290 12177 server_base.cc:1034] running on GCE node
I20260811 07:07:11.709537 12177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:11.709777 12177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:11.710939 12177 hybrid_clock.cc:648] HybridClock initialized: now 1786432031710938 us; error 39 us; skew 500 ppm
I20260811 07:07:11.712155 12177 webserver.cc:460] Webserver started at http://127.3.236.129:38605/ using document root <none> and password file <none>
I20260811 07:07:11.712432 12177 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:11.712522 12177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:11.712671 12177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:11.713798 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "cd51193ac511439dba48c4580bce84a3"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "6d9a5c08baa0ec73709f44e28cc4f36d"
server_key_iv: "c11faef52b57def26fd20db080fceba0"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.714170 12177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "cd51193ac511439dba48c4580bce84a3"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "6d9a5c08baa0ec73709f44e28cc4f36d"
server_key_iv: "c11faef52b57def26fd20db080fceba0"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.715755 12177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:11.716652 12199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.716943 12177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:11.717052 12177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "cd51193ac511439dba48c4580bce84a3"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "6d9a5c08baa0ec73709f44e28cc4f36d"
server_key_iv: "c11faef52b57def26fd20db080fceba0"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.717139 12177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:11.720532 12177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:11.720813 12177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:11.720963 12177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:11.721189 12177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:11.721530 12177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:11.721586 12177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.721626 12177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:11.721657 12177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.727480 12177 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:43433
I20260811 07:07:11.727560 12399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:43433 every 8 connection(s)
I20260811 07:07:11.727890 12177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:11.731009 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.728665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:51899 (local address 127.3.236.190:36385)
0811 07:07:11.728731 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.728734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.729002 (+ 268us) server_negotiation.cc:407] Connection header received
0811 07:07:11.729524 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.729525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.729535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.729586 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:11.730729 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:11.730731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:11.730731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:11.730736 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:11.730754 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:11.730759 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:11.730819 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.730920 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.730938 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:11.731220 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.728655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36385 (local address 127.3.236.129:51899)
0811 07:07:11.728861 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.728900 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.729400 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.729663 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.729667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.729678 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:11.730571 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:11.730572 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:11.730599 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:11.730627 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:11.730823 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:11.730829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:11.730833 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.730931 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.730996 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260811 07:07:11.731734 12400 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36385
I20260811 07:07:11.731829 12400 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:11.732017 12400 heartbeater.cc:511] Master 127.3.236.190:36385 requested a full tablet report, sending...
I20260811 07:07:11.732568 12071 ts_manager.cc:194] Registered new tserver with Master: cd51193ac511439dba48c4580bce84a3 (127.3.236.129:43433)
I20260811 07:07:11.733024 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 12177
I20260811 07:07:11.733107 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:11.733307 4018 external_mini_cluster.cc:1428] Setting key 47b07622908ac6595ab56ec8a6eed947
I20260811 07:07:11.734210 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--builtin_ntp_servers=127.3.236.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:07:11.816978 12407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:11.817252 12407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:11.817278 12407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:11.818498 12407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:11.818538 12407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:11.818565 12407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:11.820034 12407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:11.820328 12407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:11.820662 12407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:11.825852 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.825884 12413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.825913 12415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.826129 12407 server_base.cc:1034] running on GCE node
I20260811 07:07:11.826339 12407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:11.826529 12407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:11.827690 12407 hybrid_clock.cc:648] HybridClock initialized: now 1786432031827677 us; error 28 us; skew 500 ppm
I20260811 07:07:11.834728 12407 webserver.cc:460] Webserver started at http://127.3.236.130:43233/ using document root <none> and password file <none>
I20260811 07:07:11.835029 12407 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:11.835083 12407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:11.835264 12407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:11.836452 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "544a7bdb4fde681b04bea7ccc3fc465e"
server_key_iv: "e8745f18a9207247cadbb67eee2dc7a7"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.836830 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "544a7bdb4fde681b04bea7ccc3fc465e"
server_key_iv: "e8745f18a9207247cadbb67eee2dc7a7"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.838201 12407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:11.838932 12428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.839143 12407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:11.839234 12407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "544a7bdb4fde681b04bea7ccc3fc465e"
server_key_iv: "e8745f18a9207247cadbb67eee2dc7a7"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.839290 12407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:11.843003 12407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:11.843276 12407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:11.843379 12407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:11.843549 12407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:11.843801 12407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:11.843832 12407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.843851 12407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:11.843864 12407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.849838 12407 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:39491
I20260811 07:07:11.849905 12620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:39491 every 8 connection(s)
I20260811 07:07:11.850246 12407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:11.853372 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.851039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:51207 (local address 127.3.236.190:36385)
0811 07:07:11.851128 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.851130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.851385 (+ 255us) server_negotiation.cc:407] Connection header received
0811 07:07:11.851842 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.851843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.851849 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.851917 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:11.853173 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:11.853175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:11.853175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:11.853180 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:11.853196 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:11.853200 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:11.853256 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:11.853306 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 07:07:11.853318 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:11.853572 12623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.851057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36385 (local address 127.3.236.130:51207)
0811 07:07:11.851259 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.851293 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.851756 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.851983 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.851986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.851995 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:11.853023 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:11.853024 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:11.853053 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:11.853082 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:11.853257 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:11.853259 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:11.853262 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:11.853312 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 07:07:11.853361 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260811 07:07:11.854090 12621 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36385
I20260811 07:07:11.854185 12621 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:11.854302 12621 heartbeater.cc:511] Master 127.3.236.190:36385 requested a full tablet report, sending...
I20260811 07:07:11.854636 12071 ts_manager.cc:194] Registered new tserver with Master: 80ed824eb87b4bcaa1dae2d3a8dcb1f2 (127.3.236.130:39491)
I20260811 07:07:11.859848 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 12407
I20260811 07:07:11.859964 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:11.860174 4018 external_mini_cluster.cc:1428] Setting key 7e6051f165f442312e948de6e9d66c74
I20260811 07:07:11.861320 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--builtin_ntp_servers=127.3.236.148:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:07:11.971096 12624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:11.971307 12624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:11.971330 12624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:11.972535 12624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:11.972575 12624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:11.972601 12624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:11.973886 12624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:11.974071 12624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:11.974311 12624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:11.979349 12639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:11.979342 12641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.979607 12624 server_base.cc:1034] running on GCE node
W20260811 07:07:11.979341 12643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:11.979930 12624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:11.980203 12624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:11.981395 12624 hybrid_clock.cc:648] HybridClock initialized: now 1786432031981375 us; error 47 us; skew 500 ppm
I20260811 07:07:11.982674 12624 webserver.cc:460] Webserver started at http://127.3.236.131:44425/ using document root <none> and password file <none>
I20260811 07:07:11.982916 12624 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:11.982990 12624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:11.983165 12624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:11.984357 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "51619af6f3454db799ff12328d4c65af"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "50556d77515d93e014a90fc9b1c4fa94"
server_key_iv: "8b38abccb8b04757533e453bee0d5857"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.984733 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "51619af6f3454db799ff12328d4c65af"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "50556d77515d93e014a90fc9b1c4fa94"
server_key_iv: "8b38abccb8b04757533e453bee0d5857"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.986078 12624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:11.986840 12650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.987042 12624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:11.987162 12624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "51619af6f3454db799ff12328d4c65af"
format_stamp: "Formatted at 2026-08-11 07:07:11 on dist-test-slave-2d19"
server_key: "50556d77515d93e014a90fc9b1c4fa94"
server_key_iv: "8b38abccb8b04757533e453bee0d5857"
server_key_version: "encryptionkey@0"
I20260811 07:07:11.987237 12624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:11.990680 12624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:11.990949 12624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:11.991097 12624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:11.991307 12624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:11.991629 12624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:11.991685 12624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.991724 12624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:11.991751 12624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:11.997684 12624 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:42573
I20260811 07:07:11.997754 12845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:42573 every 8 connection(s)
I20260811 07:07:11.998128 12624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:12.001034 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 12624
I20260811 07:07:12.001155 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:12.001353 4018 external_mini_cluster.cc:1428] Setting key 7a7f475d7b77b9ca3e8325e39beed0be
I20260811 07:07:12.001489 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.998947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:41719 (local address 127.3.236.190:36385)
0811 07:07:11.999017 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:11.999024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:11.999266 (+ 242us) server_negotiation.cc:407] Connection header received
0811 07:07:11.999754 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:11.999755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:11.999761 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:11.999817 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.001049 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.001051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.001051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.001057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.001076 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.001081 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.001160 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.001395 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.001414 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:12.001690 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:11.998936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36385 (local address 127.3.236.131:41719)
0811 07:07:11.999148 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:11.999182 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:11.999655 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:11.999874 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:11.999877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:11.999886 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.000883 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.000883 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.000913 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.000944 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.001275 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.001282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.001285 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.001386 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.001453 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260811 07:07:12.002262 12846 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36385
I20260811 07:07:12.002372 12846 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:12.002529 12846 heartbeater.cc:511] Master 127.3.236.190:36385 requested a full tablet report, sending...
I20260811 07:07:12.002871 12071 ts_manager.cc:194] Registered new tserver with Master: 51619af6f3454db799ff12328d4c65af (127.3.236.131:42573)
I20260811 07:07:12.013296 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:07:12.015877 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.014732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58718 (local address 127.3.236.190:36385)
0811 07:07:12.014794 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.014796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.014911 (+ 115us) server_negotiation.cc:407] Connection header received
0811 07:07:12.014998 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.014999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.015005 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.015078 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.015645 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.015646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.015647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.015651 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.015668 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.015672 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.015726 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.015787 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.015804 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 07:07:12.017521 12071 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:12.018986 12071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:12.023353 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.3.236.129:43433)
0811 07:07:12.021671 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.021676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.021688 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:12.021718 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.021720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.021729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.021807 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.022971 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.022980 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.022980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.022985 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.023024 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.023051 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.023124 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.023242 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.023258 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":93,"threads_started":1}
I20260811 07:07:12.023358 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:39491 (local address 127.0.0.1:41120)
0811 07:07:12.021717 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.021729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.021775 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.022050 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.022051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.022054 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.022899 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.022900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.022905 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.022916 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.023183 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.023184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.023185 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.023216 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.023231 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260811 07:07:12.023453 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:42573 (local address 127.0.0.1:35928)
0811 07:07:12.021461 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.021473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.021518 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.022165 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.022167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.022170 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.023074 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.023075 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.023079 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.023089 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.023327 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.023327 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.023328 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.023356 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.023367 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":129,"threads_started":1}
I20260811 07:07:12.023564 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.3.236.130:39491)
0811 07:07:12.021884 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.021889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.021899 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:12.021924 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.021926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.021936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.022246 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.023020 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.023029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.023030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.023034 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.023073 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.023104 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.023403 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.023458 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.023478 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260811 07:07:12.023662 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.3.236.131:42573)
0811 07:07:12.022009 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.022014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.022023 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:12.022048 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.022050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.022059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.022302 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.023177 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.023186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.023186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.023191 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.023226 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.023250 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.023532 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.023555 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.023573 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260811 07:07:12.023345 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.021158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:43433 (local address 127.0.0.1:52478)
0811 07:07:12.021364 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.021407 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.021466 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.022002 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.022011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.022018 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.022792 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.022792 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.022806 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.022858 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.023117 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.023124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.023128 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.023227 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.023241 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
I20260811 07:07:12.024461 12723 tablet_service.cc:1467] Processing CreateTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 (DEFAULT_TABLE table=test-table [id=9931c936aa6244a8b666d798d74969c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:12.024494 12507 tablet_service.cc:1467] Processing CreateTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 (DEFAULT_TABLE table=test-table [id=9931c936aa6244a8b666d798d74969c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:12.024828 12507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbd3f653d0a46c08800afbf3e3930d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.024835 12723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbd3f653d0a46c08800afbf3e3930d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.027452 12870 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Bootstrap starting.
I20260811 07:07:12.027591 12873 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Bootstrap starting.
I20260811 07:07:12.028311 12873 tablet_bootstrap.cc:654] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.028328 12870 tablet_bootstrap.cc:654] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.028318 12283 tablet_service.cc:1467] Processing CreateTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 (DEFAULT_TABLE table=test-table [id=9931c936aa6244a8b666d798d74969c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:12.028672 12873 log.cc:824] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.028672 12870 log.cc:824] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.028718 12283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbd3f653d0a46c08800afbf3e3930d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.029529 12870 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: No bootstrap required, opened a new log
I20260811 07:07:12.029609 12870 ts_tablet_manager.cc:1397] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 07:07:12.029752 12873 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: No bootstrap required, opened a new log
I20260811 07:07:12.029816 12873 ts_tablet_manager.cc:1397] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:07:12.031577 12881 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: Bootstrap starting.
I20260811 07:07:12.032322 12881 tablet_bootstrap.cc:654] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.032408 12870 raft_consensus.cc:348] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.032501 12873 raft_consensus.cc:348] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.032603 12870 raft_consensus.cc:374] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.032624 12873 raft_consensus.cc:374] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.032652 12870 raft_consensus.cc:729] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ed824eb87b4bcaa1dae2d3a8dcb1f2, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.032662 12873 raft_consensus.cc:729] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51619af6f3454db799ff12328d4c65af, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.032666 12881 log.cc:824] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.032796 12873 consensus_queue.cc:260] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.032796 12870 consensus_queue.cc:260] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.033073 12870 ts_tablet_manager.cc:1428] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 07:07:12.033073 12873 ts_tablet_manager.cc:1428] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 07:07:12.033123 12621 heartbeater.cc:503] Master 127.3.236.190:36385 was elected leader, sending a full tablet report...
I20260811 07:07:12.033401 12846 heartbeater.cc:503] Master 127.3.236.190:36385 was elected leader, sending a full tablet report...
I20260811 07:07:12.033675 12881 tablet_bootstrap.cc:492] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: No bootstrap required, opened a new log
I20260811 07:07:12.033746 12881 ts_tablet_manager.cc:1397] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:07:12.035866 12881 raft_consensus.cc:348] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.036002 12881 raft_consensus.cc:374] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.036026 12881 raft_consensus.cc:729] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd51193ac511439dba48c4580bce84a3, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.036103 12881 consensus_queue.cc:260] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.036382 12400 heartbeater.cc:503] Master 127.3.236.190:36385 was elected leader, sending a full tablet report...
I20260811 07:07:12.036382 12881 ts_tablet_manager.cc:1428] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:07:12.072180 12886 raft_consensus.cc:482] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:12.072398 12886 raft_consensus.cc:504] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.072835 12886 leader_election.cc:288] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd51193ac511439dba48c4580bce84a3 (127.3.236.129:43433), 80ed824eb87b4bcaa1dae2d3a8dcb1f2 (127.3.236.130:39491)
I20260811 07:07:12.074152 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.073035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:43433 (local address 127.3.236.131:39397)
0811 07:07:12.073087 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.073108 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.073151 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.073333 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.073334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.073337 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.073799 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.073800 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.073808 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.073837 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.073998 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.073999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.074000 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.074052 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.074067 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 07:07:12.074182 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.073139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:39397 (local address 127.3.236.129:43433)
0811 07:07:12.073191 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.073193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.073203 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:12.073232 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.073233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.073239 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.073303 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.073911 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.073913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.073913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.073918 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.073936 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.073940 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.073999 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.074076 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.074091 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:07:12.074436 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.073108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:53719 (local address 127.3.236.130:39491)
0811 07:07:12.073152 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.073154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.073363 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:07:12.073399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.073400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.073406 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.073450 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.074163 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.074165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.074165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.074169 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.074188 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.074193 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.074256 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.074359 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.074373 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:07:12.074579 12318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbd3f653d0a46c08800afbf3e3930d0" candidate_uuid: "51619af6f3454db799ff12328d4c65af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd51193ac511439dba48c4580bce84a3" is_pre_election: true
I20260811 07:07:12.074641 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.073038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:39491 (local address 127.3.236.131:53719)
0811 07:07:12.073267 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.073274 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.073337 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.073478 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.073479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.073481 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:12.074050 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:12.074050 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:12.074057 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:12.074067 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:12.074289 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:12.074291 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:12.074292 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:07:12.074537 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 07:07:12.074553 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":66,"thread_start_us":111,"threads_started":1}
I20260811 07:07:12.074764 12318 raft_consensus.cc:2454] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51619af6f3454db799ff12328d4c65af in term 0.
I20260811 07:07:12.075039 12547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbd3f653d0a46c08800afbf3e3930d0" candidate_uuid: "51619af6f3454db799ff12328d4c65af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" is_pre_election: true
I20260811 07:07:12.075189 12547 raft_consensus.cc:2454] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51619af6f3454db799ff12328d4c65af in term 0.
I20260811 07:07:12.075434 12657 leader_election.cc:302] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51619af6f3454db799ff12328d4c65af, 80ed824eb87b4bcaa1dae2d3a8dcb1f2; no voters:
I20260811 07:07:12.075583 12886 raft_consensus.cc:2781] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:12.075652 12886 raft_consensus.cc:482] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:12.075670 12886 raft_consensus.cc:3037] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.076445 12886 raft_consensus.cc:504] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.076583 12886 leader_election.cc:288] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [CANDIDATE]: Term 1 election: Requested vote from peers cd51193ac511439dba48c4580bce84a3 (127.3.236.129:43433), 80ed824eb87b4bcaa1dae2d3a8dcb1f2 (127.3.236.130:39491)
I20260811 07:07:12.076754 12318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbd3f653d0a46c08800afbf3e3930d0" candidate_uuid: "51619af6f3454db799ff12328d4c65af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd51193ac511439dba48c4580bce84a3"
I20260811 07:07:12.076772 12547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbd3f653d0a46c08800afbf3e3930d0" candidate_uuid: "51619af6f3454db799ff12328d4c65af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2"
I20260811 07:07:12.076839 12318 raft_consensus.cc:3037] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.076860 12547 raft_consensus.cc:3037] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.077638 12318 raft_consensus.cc:2454] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51619af6f3454db799ff12328d4c65af in term 1.
I20260811 07:07:12.077646 12547 raft_consensus.cc:2454] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51619af6f3454db799ff12328d4c65af in term 1.
I20260811 07:07:12.077839 12655 leader_election.cc:302] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51619af6f3454db799ff12328d4c65af, cd51193ac511439dba48c4580bce84a3; no voters:
I20260811 07:07:12.077941 12886 raft_consensus.cc:2781] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:12.078092 12886 raft_consensus.cc:686] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 1 LEADER]: Becoming Leader. State: Replica: 51619af6f3454db799ff12328d4c65af, State: Running, Role: LEADER
I20260811 07:07:12.078190 12886 consensus_queue.cc:237] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } }
I20260811 07:07:12.079123 12068 catalog_manager.cc:5495] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af reported cstate change: term changed from 0 to 1, leader changed from <none> to 51619af6f3454db799ff12328d4c65af (127.3.236.131). New cstate: current_term: 1 leader_uuid: "51619af6f3454db799ff12328d4c65af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51619af6f3454db799ff12328d4c65af" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:12.086905 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.085930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.3.236.131:42573)
0811 07:07:12.085990 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.085992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.085999 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:07:12.086089 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.086098 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.086110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.086157 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.086761 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.086762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.086762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.086765 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.086775 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.086778 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.086816 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.086848 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.086858 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:12.088836 12318 raft_consensus.cc:1261] T bcbd3f653d0a46c08800afbf3e3930d0 P cd51193ac511439dba48c4580bce84a3 [term 1 FOLLOWER]: Refusing update from remote peer 51619af6f3454db799ff12328d4c65af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:12.088852 12547 raft_consensus.cc:1261] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 1 FOLLOWER]: Refusing update from remote peer 51619af6f3454db799ff12328d4c65af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:12.089175 12886 consensus_queue.cc:1035] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ed824eb87b4bcaa1dae2d3a8dcb1f2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 39491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:12.089324 12891 consensus_queue.cc:1035] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd51193ac511439dba48c4580bce84a3" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:12.092298 12896 mvcc.cc:205] Tried to move back new op lower bound from 7317225603432636416 to 7317225603393150976. Current Snapshot: MvccSnapshot[applied={T|T < 7317225603432636416}]
I20260811 07:07:12.094386 12071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:12.094491 12071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:12.095244 12071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96fd1f84245c4df4a3a0a3b1c7154b35: Sending DeleteTablet for 3 replicas of tablet bcbd3f653d0a46c08800afbf3e3930d0
I20260811 07:07:12.095726 12723 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:12 UTC) from {username='slave'} at 127.0.0.1:35928
I20260811 07:07:12.095742 12507 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:12 UTC) from {username='slave'} at 127.0.0.1:41120
I20260811 07:07:12.095950 12283 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbd3f653d0a46c08800afbf3e3930d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:12 UTC) from {username='slave'} at 127.0.0.1:52478
I20260811 07:07:12.095990 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 12177
I20260811 07:07:12.106571 12914 tablet_replica.cc:330] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: stopping tablet replica
I20260811 07:07:12.106719 12914 raft_consensus.cc:2229] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:12.106839 12914 raft_consensus.cc:2258] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:12.107030 12913 tablet_replica.cc:330] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: stopping tablet replica
I20260811 07:07:12.107131 12913 raft_consensus.cc:2229] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:12.107223 12913 raft_consensus.cc:2258] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:12.108810 12913 ts_tablet_manager.cc:1905] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:12.111402 12913 ts_tablet_manager.cc:1918] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:12.111493 12913 log.cc:1196] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/bcbd3f653d0a46c08800afbf3e3930d0
I20260811 07:07:12.111788 12913 ts_tablet_manager.cc:1939] T bcbd3f653d0a46c08800afbf3e3930d0 P 51619af6f3454db799ff12328d4c65af: Deleting consensus metadata
I20260811 07:07:12.112452 12049 catalog_manager.cc:4841] TS 51619af6f3454db799ff12328d4c65af (127.3.236.131:42573): tablet bcbd3f653d0a46c08800afbf3e3930d0 (table test-table [id=9931c936aa6244a8b666d798d74969c4]) successfully deleted
I20260811 07:07:12.116962 12914 ts_tablet_manager.cc:1905] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:12.119511 12914 ts_tablet_manager.cc:1918] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:12.119611 12914 log.cc:1196] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/bcbd3f653d0a46c08800afbf3e3930d0
I20260811 07:07:12.119907 12914 ts_tablet_manager.cc:1939] T bcbd3f653d0a46c08800afbf3e3930d0 P 80ed824eb87b4bcaa1dae2d3a8dcb1f2: Deleting consensus metadata
I20260811 07:07:12.120546 12049 catalog_manager.cc:4841] TS 80ed824eb87b4bcaa1dae2d3a8dcb1f2 (127.3.236.130:39491): tablet bcbd3f653d0a46c08800afbf3e3930d0 (table test-table [id=9931c936aa6244a8b666d798d74969c4]) successfully deleted
W20260811 07:07:12.123493 12046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:43433 (error 108)
I20260811 07:07:12.124074 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.123918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:43433 (local address 127.0.0.1:52494)
0811 07:07:12.123983 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.124035 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:07:12.124250 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 12407
W20260811 07:07:12.124332 12046 catalog_manager.cc:4584] TS cd51193ac511439dba48c4580bce84a3 (127.3.236.129:43433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcbd3f653d0a46c08800afbf3e3930d0: Network error: Client connection negotiation failed: client connection to 127.3.236.129:43433: connect: Connection refused (error 111)
I20260811 07:07:12.151921 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 12624
I20260811 07:07:12.165863 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 12022
2026-08-11T07:07:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (780 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 07:07:12.198242 4018 mini_oidc.cc:152] Starting JWKS server
I20260811 07:07:12.199518 4018 webserver.cc:460] Webserver started at https://127.0.0.1:38599/ using document root <none> and password file <none>
I20260811 07:07:12.199644 4018 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:07:12.199834 4018 webserver.cc:460] Webserver started at http://127.0.0.1:34165/ using document root <none> and password file <none>
2026-08-11T07:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:12Z Disabled control of system clock
I20260811 07:07:12.208778 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:39359
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:36189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:39359
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38599/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/testchainca.cert with env {}
W20260811 07:07:12.293656 12928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:12.293869 12928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:12.293891 12928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:12.295076 12928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:12.295116 12928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:12.295130 12928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:12.295142 12928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:12.295189 12928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:07:12.295207 12928 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:07:12.295229 12928 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38599/jwks.json
I20260811 07:07:12.296802 12928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:39359
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:39359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38599/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:12.297060 12928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:12.297300 12928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:12.302265 12933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.302297 12934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.302271 12937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.302443 12928 server_base.cc:1034] running on GCE node
I20260811 07:07:12.302727 12928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:12.303062 12928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:12.304220 12928 hybrid_clock.cc:648] HybridClock initialized: now 1786432032304206 us; error 27 us; skew 500 ppm
I20260811 07:07:12.305594 12928 webserver.cc:460] Webserver started at http://127.3.236.190:33679/ using document root <none> and password file <none>
I20260811 07:07:12.305846 12928 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:12.305896 12928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:12.306066 12928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:12.307135 12928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "022a6398ce12444c8837e79a63aa6bd0"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "c3ebe94e802ece6c87bed1824139cd63"
server_key_iv: "be8068774b2f7e5514a15babebab01c5"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.307507 12928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "022a6398ce12444c8837e79a63aa6bd0"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "c3ebe94e802ece6c87bed1824139cd63"
server_key_iv: "be8068774b2f7e5514a15babebab01c5"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.308920 12928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 07:07:12.309662 12947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:12.309830 12928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:12.309935 12928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "022a6398ce12444c8837e79a63aa6bd0"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "c3ebe94e802ece6c87bed1824139cd63"
server_key_iv: "be8068774b2f7e5514a15babebab01c5"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.310037 12928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:12.321552 12928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:12.321874 12928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:12.322029 12928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:12.326152 12928 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:39359
I20260811 07:07:12.326195 13035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:39359 every 8 connection(s)
I20260811 07:07:12.326538 12928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:12.327567 13036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.329941 13036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0: Bootstrap starting.
I20260811 07:07:12.330533 13036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.330835 13036 log.cc:824] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.331519 13036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0: No bootstrap required, opened a new log
I20260811 07:07:12.333130 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 12928
I20260811 07:07:12.333253 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:12.333204 13036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } }
I20260811 07:07:12.333417 13036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.333467 13036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022a6398ce12444c8837e79a63aa6bd0, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.333472 4018 external_mini_cluster.cc:1428] Setting key e9c1c364aa04e446ad94fba86b13e749
I20260811 07:07:12.333614 13036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } }
I20260811 07:07:12.333716 13036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:12.333760 13036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:12.333876 13036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.334568 13036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } }
I20260811 07:07:12.334692 13036 leader_election.cc:302] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 022a6398ce12444c8837e79a63aa6bd0; no voters:
I20260811 07:07:12.334873 13036 leader_election.cc:288] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:12.334933 13051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:12.335100 13051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 022a6398ce12444c8837e79a63aa6bd0, State: Running, Role: LEADER
I20260811 07:07:12.335147 13036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:12.335279 13051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } }
I20260811 07:07:12.335645 13051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022a6398ce12444c8837e79a63aa6bd0. Latest consensus state: current_term: 1 leader_uuid: "022a6398ce12444c8837e79a63aa6bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } } }
I20260811 07:07:12.335659 13052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "022a6398ce12444c8837e79a63aa6bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022a6398ce12444c8837e79a63aa6bd0" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 39359 } } }
I20260811 07:07:12.335809 13051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:12.335820 13052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:12.336501 13059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:12.337026 13059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:12.338630 13059 catalog_manager.cc:1347] Generated new cluster ID: e7885544a0fc421aa7e3615941d6e191
I20260811 07:07:12.338971 13059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:12.339990 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.333948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.3.236.190:39359)
0811 07:07:12.334187 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.334203 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.334229 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:07:12.334320 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.334325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.334384 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.334513 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.334751 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.336574 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.337659 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.338285 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.338406 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.339072 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.339076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.339076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.339280 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.339323 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.339339 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.339663 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.339696 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.339775 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260811 07:07:12.341320 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:07:12.349709 13059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:12.350239 13059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:12.354621 13059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 022a6398ce12444c8837e79a63aa6bd0: Generated new TSK 0
I20260811 07:07:12.354810 13059 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 07:07:12.387094 13088 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 07:07:12.387234 13088 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 07:07:12.387569 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.368141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34646 (local address 127.3.236.190:39359)
0811 07:07:12.368217 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.368219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.368412 (+ 193us) server_negotiation.cc:407] Connection header received
0811 07:07:12.368545 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.368546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.368561 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.368638 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:07:12.368810 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.369304 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.370122 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.370306 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.370379 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.370597 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:07:12.387321 (+ 16724us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 07:07:12.387499 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 07:07:12.387632 13050 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 07:07:12.387933 13086 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:12.368072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:39359 (local address 127.0.0.1:34646)
0811 07:07:12.368341 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.368350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.368436 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.368651 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.368652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.368654 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:07:12.368736 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.368738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.369393 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.369394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.370018 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.370020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.370382 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.370383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.370451 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.370479 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:07:12.387465 (+ 16986us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 07:07:12.387573 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.236.190:39359: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.236.190:39359: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 07:07:12.388509 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 12928
2026-08-11T07:07:12Z chronyd exiting
I20260811 07:07:12.436798 4018 test_util.cc:175] -----------------------------------------------
I20260811 07:07:12.436880 4018 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786432006544353-4018-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 07:07:12.437638 4018 mini_oidc.cc:152] Starting JWKS server
I20260811 07:07:12.438294 4018 webserver.cc:460] Webserver started at https://127.0.0.1:43219/ using document root <none> and password file <none>
I20260811 07:07:12.438338 4018 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:07:12.438479 4018 webserver.cc:460] Webserver started at http://127.0.0.1:43833/ using document root <none> and password file <none>
2026-08-11T07:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:12Z Disabled control of system clock
I20260811 07:07:12.447321 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42893
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42893
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43219/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/testchainca.cert with env {}
W20260811 07:07:12.530599 13098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:12.530864 13098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:12.530913 13098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:12.532135 13098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:12.532212 13098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:12.532233 13098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:12.532254 13098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:12.532272 13098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:07:12.532357 13098 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:07:12.532379 13098 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43219/jwks.json
I20260811 07:07:12.533969 13098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43219/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:12.534257 13098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:12.534489 13098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:12.539587 13110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.539558 13114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.539593 13116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.539748 13098 server_base.cc:1034] running on GCE node
I20260811 07:07:12.540033 13098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:12.540356 13098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:12.541504 13098 hybrid_clock.cc:648] HybridClock initialized: now 1786432032541482 us; error 39 us; skew 500 ppm
I20260811 07:07:12.542697 13098 webserver.cc:460] Webserver started at http://127.3.236.190:34381/ using document root <none> and password file <none>
I20260811 07:07:12.542936 13098 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:12.542982 13098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:12.543151 13098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:12.544193 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "8ef7b7f00b8646be85ae2ce8749101d8"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "fa5f2fff2d2d72c129113a184aacb1bc"
server_key_iv: "7037c0e165435b8bfa7d9d05397abf12"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.544663 13098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "8ef7b7f00b8646be85ae2ce8749101d8"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "fa5f2fff2d2d72c129113a184aacb1bc"
server_key_iv: "7037c0e165435b8bfa7d9d05397abf12"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.545991 13098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:12.546734 13126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:12.547022 13098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:12.547127 13098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "8ef7b7f00b8646be85ae2ce8749101d8"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "fa5f2fff2d2d72c129113a184aacb1bc"
server_key_iv: "7037c0e165435b8bfa7d9d05397abf12"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.547210 13098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:12.564364 13098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:12.564734 13098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:12.564903 13098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:12.569099 13098 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:42893
I20260811 07:07:12.569152 13216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:42893 every 8 connection(s)
I20260811 07:07:12.569525 13098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:12.570554 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.572904 13218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8: Bootstrap starting.
I20260811 07:07:12.573479 13218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.573760 13218 log.cc:824] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.573843 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 13098
I20260811 07:07:12.573968 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:12.574151 4018 external_mini_cluster.cc:1428] Setting key d07505d5070758eb033b103260869b96
I20260811 07:07:12.574546 13218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8: No bootstrap required, opened a new log
I20260811 07:07:12.577149 13218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } }
I20260811 07:07:12.577337 13218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.577378 13218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef7b7f00b8646be85ae2ce8749101d8, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.577517 13218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } }
I20260811 07:07:12.577600 13218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:12.577630 13218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:12.577678 13218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.578369 13218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } }
I20260811 07:07:12.578493 13218 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef7b7f00b8646be85ae2ce8749101d8; no voters:
I20260811 07:07:12.578687 13218 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:12.578708 13225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:12.578821 13225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef7b7f00b8646be85ae2ce8749101d8, State: Running, Role: LEADER
I20260811 07:07:12.578959 13218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:12.578981 13225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } }
I20260811 07:07:12.579392 13231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } } }
I20260811 07:07:12.579632 13231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:12.579555 13230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef7b7f00b8646be85ae2ce8749101d8. Latest consensus state: current_term: 1 leader_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef7b7f00b8646be85ae2ce8749101d8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42893 } } }
I20260811 07:07:12.579923 13230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:12.580103 13243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:12.580578 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.574683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.3.236.190:42893)
0811 07:07:12.574908 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.574916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.574940 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:07:12.575817 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.575826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.575885 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.576001 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.576317 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.577167 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.578298 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.578861 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.579329 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.579719 (+ 390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.579723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.579723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.579930 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.579973 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.579987 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.580084 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.580168 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.580377 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260811 07:07:12.580786 13243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:12.581526 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:07:12.582774 13243 catalog_manager.cc:1347] Generated new cluster ID: b288018cca6d4d49a9dda122e795239c
I20260811 07:07:12.582832 13243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:12.595486 13243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:12.596179 13243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:12.601190 13243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef7b7f00b8646be85ae2ce8749101d8: Generated new TSK 0
I20260811 07:07:12.601375 13243 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 07:07:12.626355 13274 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 07:07:12.626447 13274 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 07:07:12.626806 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.612370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.3.236.190:42893)
0811 07:07:12.612441 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.612444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.612549 (+ 105us) server_negotiation.cc:407] Connection header received
0811 07:07:12.612592 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.612593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.612606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.612689 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:07:12.612805 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.613604 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.614380 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.614701 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.614779 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.614977 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:07:12.626563 (+ 11586us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 07:07:12.626724 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50662: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 07:07:12.626848 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:12.612320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42893 (local address 127.0.0.1:50662)
0811 07:07:12.612467 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.612478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.612542 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.612693 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.612694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.612696 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:07:12.612749 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.612751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.613699 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.613700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.614292 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.614293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.614785 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.614786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.614823 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.614828 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:07:12.626673 (+ 11845us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 07:07:12.626775 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.236.190:42893: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260811 07:07:12.626874 13223 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50662: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 07:07:12.627167 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 13098
2026-08-11T07:07:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 07:07:12.676501 4018 mini_oidc.cc:152] Starting JWKS server
I20260811 07:07:12.677258 4018 webserver.cc:460] Webserver started at https://127.0.0.1:33221/ using document root <none> and password file <none>
I20260811 07:07:12.677305 4018 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:07:12.677475 4018 webserver.cc:460] Webserver started at http://127.0.0.1:42659/ using document root <none> and password file <none>
2026-08-11T07:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:12Z Disabled control of system clock
I20260811 07:07:12.686522 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43723
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:33317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:43723
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33221/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 07:07:12.768926 13281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:12.769150 13281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:12.769172 13281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:12.770285 13281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:12.770361 13281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:12.770383 13281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:12.770406 13281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:12.770434 13281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:07:12.770467 13281 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:07:12.770486 13281 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33221/jwks.json
I20260811 07:07:12.771860 13281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:43723
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33221/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:12.772171 13281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:12.772497 13281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:12.777387 13289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.777413 13281 server_base.cc:1034] running on GCE node
W20260811 07:07:12.777424 13290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.777387 13300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.777812 13281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:12.778119 13281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:12.779284 13281 hybrid_clock.cc:648] HybridClock initialized: now 1786432032779266 us; error 28 us; skew 500 ppm
I20260811 07:07:12.780582 13281 webserver.cc:460] Webserver started at http://127.3.236.190:33931/ using document root <none> and password file <none>
I20260811 07:07:12.780854 13281 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:12.780928 13281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:12.781086 13281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:12.782153 13281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "23de409ab6634a1b90f1617236b7fc92"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "975d290fe20d1aecd2935fa421c85dde"
server_key_iv: "47419627e0fe9e0a5669ccee8c6fa423"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.782524 13281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "23de409ab6634a1b90f1617236b7fc92"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "975d290fe20d1aecd2935fa421c85dde"
server_key_iv: "47419627e0fe9e0a5669ccee8c6fa423"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.783808 13281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:12.784590 13310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:12.784758 13281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:12.784852 13281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "23de409ab6634a1b90f1617236b7fc92"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "975d290fe20d1aecd2935fa421c85dde"
server_key_iv: "47419627e0fe9e0a5669ccee8c6fa423"
server_key_version: "encryptionkey@0"
I20260811 07:07:12.784933 13281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:12.793238 13281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:12.793532 13281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:12.793689 13281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:12.797751 13281 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:43723
I20260811 07:07:12.797868 13391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:43723 every 8 connection(s)
I20260811 07:07:12.798172 13281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:12.798872 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:12.800859 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 13281
I20260811 07:07:12.800980 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:12.801178 4018 external_mini_cluster.cc:1428] Setting key bd770325c82730c6f8b9758e0be277f4
I20260811 07:07:12.802446 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: Bootstrap starting.
I20260811 07:07:12.803192 13392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:12.803526 13392 log.cc:824] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:12.804301 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: No bootstrap required, opened a new log
I20260811 07:07:12.806455 13392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } }
I20260811 07:07:12.806644 13392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:12.806689 13392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23de409ab6634a1b90f1617236b7fc92, State: Initialized, Role: FOLLOWER
I20260811 07:07:12.806829 13392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } }
I20260811 07:07:12.806923 13392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:12.806970 13392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:12.807025 13392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:12.807435 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.801735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.3.236.190:43723)
0811 07:07:12.801966 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.801984 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.802022 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:07:12.802587 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.802596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.802674 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.802841 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:12.803018 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.804437 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.805188 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.805816 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.805968 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.806587 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:12.806591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:12.806591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:12.806802 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:12.806845 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:12.806859 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:12.806949 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:12.807024 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 07:07:12.807222 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260811 07:07:12.807776 13392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } }
I20260811 07:07:12.807881 13392 leader_election.cc:302] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23de409ab6634a1b90f1617236b7fc92; no voters:
I20260811 07:07:12.808053 13392 leader_election.cc:288] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:12.808100 13406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:12.808296 13406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [term 1 LEADER]: Becoming Leader. State: Replica: 23de409ab6634a1b90f1617236b7fc92, State: Running, Role: LEADER
I20260811 07:07:12.808360 13392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:12.808391 13406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } }
I20260811 07:07:12.808497 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:07:12.808724 13406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23de409ab6634a1b90f1617236b7fc92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } } }
I20260811 07:07:12.808792 13406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:12.808951 13406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23de409ab6634a1b90f1617236b7fc92. Latest consensus state: current_term: 1 leader_uuid: "23de409ab6634a1b90f1617236b7fc92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23de409ab6634a1b90f1617236b7fc92" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43723 } } }
I20260811 07:07:12.809016 13406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92 [sys.catalog]: This master's current role is: LEADER
W20260811 07:07:12.810130 13432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:07:12.810174 13432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:07:12.810241 13418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:12.810420 13418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:12.811887 13418 catalog_manager.cc:1347] Generated new cluster ID: f7affb8edb794fc699717d5a32718144
I20260811 07:07:12.811940 13418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:12.822712 13418 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:12.823264 13418 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:12.830672 13418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23de409ab6634a1b90f1617236b7fc92: Generated new TSK 0
I20260811 07:07:12.830870 13418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:12.845131 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:12.841786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.3.236.190:43723)
0811 07:07:12.841854 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:12.841856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:12.841968 (+ 112us) server_negotiation.cc:407] Connection header received
0811 07:07:12.842050 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:12.842050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:12.842062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:12.842134 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:07:12.842281 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.842788 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.843569 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.843798 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.843886 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.844073 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:07:12.844901 (+ 828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33221/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 07:07:12.845055 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60794: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33221/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 07:07:12.845201 13401 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60794: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33221/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 07:07:12.845220 13445 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:12.841721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43723 (local address 127.0.0.1:60794)
0811 07:07:12.841901 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:12.841910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:12.841986 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:12.842139 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:12.842140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:12.842141 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:07:12.842198 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.842200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.842875 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.842876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.843481 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:12.843483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:12.843897 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:12.843899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:12.843958 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:12.843968 (+ 10us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:07:12.845011 (+ 1043us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33221/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 07:07:12.845111 (+ 100us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.236.190:43723: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33221/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260811 07:07:12.848474 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 13281
E20260811 07:07:12.864029 13447 webserver.cc:533] Webserver: sslize failed:
E20260811 07:07:12.864101 13447 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T07:07:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (218 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T07:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:12Z Disabled control of system clock
I20260811 07:07:12.903975 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44097
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:44097
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/testchainca.cert with env {}
W20260811 07:07:12.989284 13454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:12.989550 13454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:12.989599 13454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:12.990796 13454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:12.990862 13454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:12.990882 13454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:12.990919 13454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:12.990950 13454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:07:12.990981 13454 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:07:12.991007 13454 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 07:07:12.992594 13454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:44097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:12.992883 13454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:12.993112 13454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:12.998252 13468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.998322 13454 server_base.cc:1034] running on GCE node
W20260811 07:07:12.998279 13461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:12.998279 13460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:12.998704 13454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:12.999066 13454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:13.000244 13454 hybrid_clock.cc:648] HybridClock initialized: now 1786432033000224 us; error 39 us; skew 500 ppm
I20260811 07:07:13.001614 13454 webserver.cc:460] Webserver started at http://127.3.236.190:42827/ using document root <none> and password file <none>
I20260811 07:07:13.001881 13454 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:13.001955 13454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:13.002111 13454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:13.003175 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "8aa28309a950409eb4d45ab006f34bb9"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "a59c2f7479eb2dbd0accd9364db6c253"
server_key_iv: "02fef0a6d6f282f5bac0a7be01f9b77a"
server_key_version: "encryptionkey@0"
I20260811 07:07:13.003548 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "8aa28309a950409eb4d45ab006f34bb9"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "a59c2f7479eb2dbd0accd9364db6c253"
server_key_iv: "02fef0a6d6f282f5bac0a7be01f9b77a"
server_key_version: "encryptionkey@0"
I20260811 07:07:13.004961 13454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:13.005687 13479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:13.005856 13454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:13.005949 13454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "8aa28309a950409eb4d45ab006f34bb9"
format_stamp: "Formatted at 2026-08-11 07:07:12 on dist-test-slave-2d19"
server_key: "a59c2f7479eb2dbd0accd9364db6c253"
server_key_iv: "02fef0a6d6f282f5bac0a7be01f9b77a"
server_key_version: "encryptionkey@0"
I20260811 07:07:13.006031 13454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:13.023840 13454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:13.024204 13454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:13.024407 13454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:13.028584 13454 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:44097
I20260811 07:07:13.028632 13582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:44097 every 8 connection(s)
I20260811 07:07:13.029007 13454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:13.030015 13583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:13.032389 13583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9: Bootstrap starting.
I20260811 07:07:13.032939 13583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:13.033198 13583 log.cc:824] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:13.033831 13583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9: No bootstrap required, opened a new log
I20260811 07:07:13.035449 13583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } }
I20260811 07:07:13.035585 13583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:13.035607 13583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa28309a950409eb4d45ab006f34bb9, State: Initialized, Role: FOLLOWER
I20260811 07:07:13.035710 13583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } }
I20260811 07:07:13.035761 13583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:13.035781 13583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:13.035806 13583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:13.036365 13583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } }
I20260811 07:07:13.036453 13583 leader_election.cc:302] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aa28309a950409eb4d45ab006f34bb9; no voters:
I20260811 07:07:13.036572 13583 leader_election.cc:288] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:13.036659 13588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:13.036805 13588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [term 1 LEADER]: Becoming Leader. State: Replica: 8aa28309a950409eb4d45ab006f34bb9, State: Running, Role: LEADER
I20260811 07:07:13.036845 13583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:13.036906 13588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } }
I20260811 07:07:13.037298 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aa28309a950409eb4d45ab006f34bb9. Latest consensus state: current_term: 1 leader_uuid: "8aa28309a950409eb4d45ab006f34bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } } }
I20260811 07:07:13.037382 13589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aa28309a950409eb4d45ab006f34bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa28309a950409eb4d45ab006f34bb9" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44097 } } }
I20260811 07:07:13.037396 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:13.037511 13589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:13.037719 13602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:13.038259 13602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:13.038726 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 13454
I20260811 07:07:13.038853 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:13.039067 4018 external_mini_cluster.cc:1428] Setting key 8fb6055e53c1079720e6f31c679ce879
I20260811 07:07:13.039952 13602 catalog_manager.cc:1347] Generated new cluster ID: 04202dea695f43ebafe5f60cc5ddc444
I20260811 07:07:13.040005 13602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:13.045171 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:13.039601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.3.236.190:44097)
0811 07:07:13.039788 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:13.039796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:13.039828 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:07:13.039945 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:13.039952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:13.040026 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:13.040180 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:13.040395 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.041808 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.042838 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.043482 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.043609 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:13.044330 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:13.044334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:13.044334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:13.044549 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:13.044590 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:13.044605 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:13.044677 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:13.044743 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 07:07:13.044938 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260811 07:07:13.046396 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:07:13.050628 13602 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:13.051229 13602 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:13.061386 13602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8aa28309a950409eb4d45ab006f34bb9: Generated new TSK 0
I20260811 07:07:13.061599 13602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:14.056962 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:13.067916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.3.236.190:44097)
0811 07:07:13.067978 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:13.067981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:13.068083 (+ 102us) server_negotiation.cc:407] Connection header received
0811 07:07:13.068145 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:13.068145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:13.068158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:13.068231 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:07:13.068430 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.069222 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.070023 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.070331 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.070403 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:13.070573 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:07:14.056729 (+986156us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 07:07:14.056899 (+ 170us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 07:07:14.057071 13621 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 07:07:14.057248 13633 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:13.067826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44097 (local address 127.0.0.1:48300)
0811 07:07:13.068002 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:13.068013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:13.068080 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:13.068238 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:13.068239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:13.068241 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:07:13.068338 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:13.068341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.069308 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.069309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:13.069934 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:13.069937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:13.070422 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:13.070423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:13.070468 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:13.070474 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:07:14.056967 (+986493us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 07:07:14.057110 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.236.190:44097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260811 07:07:14.057644 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 13454
2026-08-11T07:07:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1187 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:14 dist-test-slave-2d19 krb5kdc[13658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:14 dist-test-slave-2d19 krb5kdc[13658](info): set up 2 sockets
Aug 11 07:07:14 dist-test-slave-2d19 krb5kdc[13658](info): commencing operation
krb5kdc: starting...
W20260811 07:07:15.539314 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:15 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:15.667438 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:45337
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:45337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:15.749982 13679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:15.750187 13679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:15.750209 13679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:15.751340 13679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:15.751374 13679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:15.751389 13679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:15.751400 13679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:15.751412 13679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:15.752874 13679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:45337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:15.753065 13679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:15.753230 13679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:15.758342 13695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:15.758416 13679 server_base.cc:1034] running on GCE node
W20260811 07:07:15.758373 13692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:15.758363 13691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:15.758868 13679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:15.759198 13679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:15.760315 13679 hybrid_clock.cc:648] HybridClock initialized: now 1786432035760292 us; error 39 us; skew 500 ppm
Aug 11 07:07:15 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432035, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:15.762912 13679 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:15.763350 13679 webserver.cc:460] Webserver started at http://127.3.236.190:43233/ using document root <none> and password file <none>
I20260811 07:07:15.763588 13679 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:15.763667 13679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:15.763818 13679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:15.764972 13679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "e45c5fb3c1834053b2dee124c8c977e1"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "d9607c30097cb8fefabafa6c3ef50822"
server_key_iv: "b2798e72f86c5fd1be000b132c5acb9e"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.765354 13679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "e45c5fb3c1834053b2dee124c8c977e1"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "d9607c30097cb8fefabafa6c3ef50822"
server_key_iv: "b2798e72f86c5fd1be000b132c5acb9e"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.766749 13679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:15.767455 13707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:15.767630 13679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:15.767738 13679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "e45c5fb3c1834053b2dee124c8c977e1"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "d9607c30097cb8fefabafa6c3ef50822"
server_key_iv: "b2798e72f86c5fd1be000b132c5acb9e"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.767813 13679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:15.804040 13679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:15.806560 13679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:15.806742 13679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:15.810986 13679 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:45337
I20260811 07:07:15.811048 13805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:45337 every 8 connection(s)
I20260811 07:07:15.811440 13679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:15.812489 13807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:15.812891 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 13679
I20260811 07:07:15.813021 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:15.813228 4018 external_mini_cluster.cc:1428] Setting key f34a561a235692d4d090d04614df2208
I20260811 07:07:15.815663 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1: Bootstrap starting.
I20260811 07:07:15.816416 13807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:15.816718 13807 log.cc:824] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:15.817430 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1: No bootstrap required, opened a new log
I20260811 07:07:15.819558 13807 raft_consensus.cc:348] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } }
I20260811 07:07:15.819773 13807 raft_consensus.cc:374] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:15.819811 13807 raft_consensus.cc:729] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45c5fb3c1834053b2dee124c8c977e1, State: Initialized, Role: FOLLOWER
I20260811 07:07:15.819938 13807 consensus_queue.cc:260] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } }
I20260811 07:07:15.820032 13807 raft_consensus.cc:388] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:15.820072 13807 raft_consensus.cc:482] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:15.820124 13807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 07:07:15 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:15.820799 13807 raft_consensus.cc:504] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } }
I20260811 07:07:15.820921 13807 leader_election.cc:302] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e45c5fb3c1834053b2dee124c8c977e1; no voters:
I20260811 07:07:15.821099 13807 leader_election.cc:288] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:15.821185 13814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:15.821341 13814 raft_consensus.cc:686] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [term 1 LEADER]: Becoming Leader. State: Replica: e45c5fb3c1834053b2dee124c8c977e1, State: Running, Role: LEADER
I20260811 07:07:15.821424 13807 sys_catalog.cc:564] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:15.821606 13814 consensus_queue.cc:237] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } }
I20260811 07:07:15.822047 13818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e45c5fb3c1834053b2dee124c8c977e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } } }
I20260811 07:07:15.822134 13818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:15.822355 13828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:15.822355 13815 sys_catalog.cc:455] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e45c5fb3c1834053b2dee124c8c977e1. Latest consensus state: current_term: 1 leader_uuid: "e45c5fb3c1834053b2dee124c8c977e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5fb3c1834053b2dee124c8c977e1" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 45337 } } }
I20260811 07:07:15.822547 13815 sys_catalog.cc:458] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:15.822911 13828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:15.824852 13828 catalog_manager.cc:1347] Generated new cluster ID: e8042655bc5749078b31c7dee20ba685
I20260811 07:07:15.824882 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:15.813793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.3.236.190:45337)
0811 07:07:15.814009 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:15.814019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:15.814055 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:07:15.814588 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:15.814597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:15.814658 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:15.814811 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:15.815412 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:15.816769 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:15.817517 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:15.818132 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:15.818254 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:15.820790 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:15.820794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:15.820795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:15.820807 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:15.823175 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:15.823598 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:15.823601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:15.823602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:15.823704 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:15.823935 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:15.823937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:15.823937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:15.824278 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:15.824438 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:15.824550 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 07:07:15.824692 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260811 07:07:15.824934 13828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:15.828886 13828 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:15.829536 13828 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:15.831758 13828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e45c5fb3c1834053b2dee124c8c977e1: Generated new TSK 0
I20260811 07:07:15.831933 13828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:15.866616 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:15.949662 13855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:15.949873 13855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:15.949898 13855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:15.951086 13855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:15.951133 13855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:15.951160 13855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:15.952760 13855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:15.953032 13855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:15.953280 13855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:15.958341 13861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:15.958501 13866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:15.958390 13860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:15.958716 13855 server_base.cc:1034] running on GCE node
I20260811 07:07:15.959012 13855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:15.959276 13855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:15.960435 13855 hybrid_clock.cc:648] HybridClock initialized: now 1786432035960403 us; error 66 us; skew 500 ppm
Aug 11 07:07:15 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432035, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:15.963135 13855 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:15.963578 13855 webserver.cc:460] Webserver started at http://127.3.236.129:34353/ using document root <none> and password file <none>
I20260811 07:07:15.963866 13855 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:15.963935 13855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:15.964095 13855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:15.965238 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "f29488b8eb534b22a311ae3d8e8f2a66"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "37633bad205a3454f53147d5c7f34042"
server_key_iv: "3bd7bc49ea106bed2cdf514a94e0e4a5"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.965607 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "f29488b8eb534b22a311ae3d8e8f2a66"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "37633bad205a3454f53147d5c7f34042"
server_key_iv: "3bd7bc49ea106bed2cdf514a94e0e4a5"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.966974 13855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:15.967698 13876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:15.967876 13855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:15.967980 13855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "f29488b8eb534b22a311ae3d8e8f2a66"
format_stamp: "Formatted at 2026-08-11 07:07:15 on dist-test-slave-2d19"
server_key: "37633bad205a3454f53147d5c7f34042"
server_key_iv: "3bd7bc49ea106bed2cdf514a94e0e4a5"
server_key_version: "encryptionkey@0"
I20260811 07:07:15.968062 13855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:15.993019 13855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:15.995472 13855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:15.995651 13855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:15.995918 13855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:15.996304 13855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:15.996361 13855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:15.996405 13855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:15.996448 13855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.002422 13855 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:45999
I20260811 07:07:16.002477 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:45999 every 8 connection(s)
I20260811 07:07:16.002837 13855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432035, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:16.012547 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.003715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:44709 (local address 127.3.236.190:45337)
0811 07:07:16.003804 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.003807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.003980 (+ 173us) server_negotiation.cc:407] Connection header received
0811 07:07:16.005010 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.005012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.005024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.005093 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.005736 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.006258 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.007239 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.007483 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.007543 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.009745 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.009747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.009747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.009754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.011277 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.011651 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.011651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.011652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.011725 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.011959 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.011959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.011959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.012065 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.012129 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.012401 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.012475 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:07:16.012776 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.003647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45337 (local address 127.3.236.129:44709)
0811 07:07:16.003844 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:16.003882 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:16.004897 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:16.005175 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:16.005179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:16.005435 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:16.005643 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.005647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.006343 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.006344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.007147 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.007148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.007563 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.007564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.007669 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.008209 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:16.008218 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:16.009657 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:16.011354 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.011360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.011362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.011583 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.011788 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.011788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.011788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.011901 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.012123 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:16.012128 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:16.012237 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 07:07:16.012383 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 07:07:16.012507 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260811 07:07:16.012775 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 13855
I20260811 07:07:16.012887 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:16.013111 4018 external_mini_cluster.cc:1428] Setting key 1d4911870a701e7edf1b6dffedd96a68
I20260811 07:07:16.013346 14102 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45337
I20260811 07:07:16.013466 14102 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:16.013654 14102 heartbeater.cc:511] Master 127.3.236.190:45337 requested a full tablet report, sending...
I20260811 07:07:16.014251 13743 ts_manager.cc:194] Registered new tserver with Master: f29488b8eb534b22a311ae3d8e8f2a66 (127.3.236.129:45999)
I20260811 07:07:16.015142 13743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:44709
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:16.040756 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:16.125648 14112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:16.125903 14112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:16.125952 14112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:16.127121 14112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:16.127188 14112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:16.127233 14112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:16.128755 14112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:16.129031 14112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:16.129298 14112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:16.134454 14120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:16.134481 14122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:16.134487 14124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:16.134769 14112 server_base.cc:1034] running on GCE node
I20260811 07:07:16.134994 14112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:16.135205 14112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:16.136363 14112 hybrid_clock.cc:648] HybridClock initialized: now 1786432036136344 us; error 32 us; skew 500 ppm
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.139067 14112 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:16.139556 14112 webserver.cc:460] Webserver started at http://127.3.236.130:35369/ using document root <none> and password file <none>
I20260811 07:07:16.139786 14112 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:16.139832 14112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:16.139971 14112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:16.141081 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "603903bdfe6547348fa0c10bfda4aa36"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "7710629e417bb8a6206d93d3270aa167"
server_key_iv: "f6d1073cd603e45e38d5a6d159649775"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.141461 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "603903bdfe6547348fa0c10bfda4aa36"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "7710629e417bb8a6206d93d3270aa167"
server_key_iv: "f6d1073cd603e45e38d5a6d159649775"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.142817 14112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:16.143545 14137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.143733 14112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:16.143839 14112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "603903bdfe6547348fa0c10bfda4aa36"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "7710629e417bb8a6206d93d3270aa167"
server_key_iv: "f6d1073cd603e45e38d5a6d159649775"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.143914 14112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:16.155725 14112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:16.158295 14112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:16.158473 14112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:16.158727 14112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:16.159058 14112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:16.159114 14112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.159157 14112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:16.159185 14112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.165642 14112 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:40171
I20260811 07:07:16.165688 14327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:40171 every 8 connection(s)
I20260811 07:07:16.166085 14112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:16.176205 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 14112
I20260811 07:07:16.176350 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:16.176698 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.166920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:41261 (local address 127.3.236.190:45337)
0811 07:07:16.166996 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.166998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.167207 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:07:16.168245 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.168246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.168287 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.168350 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.169070 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.169580 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.170798 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.171169 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.171305 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.173826 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.173828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.173828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.173835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.175175 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.175555 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.175556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.175556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.175612 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.175946 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.175947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.175947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.176070 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.176125 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.176534 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.176616 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:07:16.176712 4018 external_mini_cluster.cc:1428] Setting key 5d3a48b46b51928c0a47b9f90d208b4d
I20260811 07:07:16.177080 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.166902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45337 (local address 127.3.236.130:41261)
0811 07:07:16.167076 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:16.167116 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:16.168115 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:16.168433 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:16.168437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:16.168725 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:16.168925 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.168947 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.169693 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.169694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.170685 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.170687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.171319 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.171320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.171473 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.172046 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:16.172057 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:16.173732 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:16.175252 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.175258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.175260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.175488 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.175717 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.175718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.175718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.175849 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.176146 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:16.176155 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:16.176366 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 07:07:16.176497 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 07:07:16.176823 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260811 07:07:16.177659 14328 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45337
I20260811 07:07:16.177778 14328 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:16.177985 14328 heartbeater.cc:511] Master 127.3.236.190:45337 requested a full tablet report, sending...
I20260811 07:07:16.178433 13743 ts_manager.cc:194] Registered new tserver with Master: 603903bdfe6547348fa0c10bfda4aa36 (127.3.236.130:40171)
I20260811 07:07:16.178815 13743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:41261
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:16.205436 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:16.290313 14333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:16.290537 14333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:16.290563 14333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:16.291664 14333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:16.291702 14333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:16.291728 14333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:16.293200 14333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:16.293397 14333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:16.293596 14333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:16.298763 14339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:16.298802 14340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:16.298873 14343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:16.298985 14333 server_base.cc:1034] running on GCE node
I20260811 07:07:16.299226 14333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:16.299467 14333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:16.300616 14333 hybrid_clock.cc:648] HybridClock initialized: now 1786432036300594 us; error 37 us; skew 500 ppm
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.303160 14333 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:16.303632 14333 webserver.cc:460] Webserver started at http://127.3.236.131:34953/ using document root <none> and password file <none>
I20260811 07:07:16.303879 14333 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:16.303956 14333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:16.304109 14333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:16.305217 14333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "b6627fea0de54ec7aac319621edc7011"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "e491554f2539dd3549394b46390989e9"
server_key_iv: "076a72091e29e396f6e234fdecf1af75"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.305588 14333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "b6627fea0de54ec7aac319621edc7011"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "e491554f2539dd3549394b46390989e9"
server_key_iv: "076a72091e29e396f6e234fdecf1af75"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.306905 14333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:16.307662 14360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.307876 14333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:16.307993 14333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "b6627fea0de54ec7aac319621edc7011"
format_stamp: "Formatted at 2026-08-11 07:07:16 on dist-test-slave-2d19"
server_key: "e491554f2539dd3549394b46390989e9"
server_key_iv: "076a72091e29e396f6e234fdecf1af75"
server_key_version: "encryptionkey@0"
I20260811 07:07:16.308073 14333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:16.329010 14333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:16.331256 14333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:16.331434 14333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:16.331688 14333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:16.332089 14333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:16.332129 14333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.332183 14333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:16.332213 14333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:16.338312 14333 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:40899
I20260811 07:07:16.338357 14542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:40899 every 8 connection(s)
I20260811 07:07:16.338730 14333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:16.341336 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 14333
I20260811 07:07:16.341430 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:16.341612 4018 external_mini_cluster.cc:1428] Setting key cebb7f650f13f71f6313616c1323a3c3
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:16.349013 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.339498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:57985 (local address 127.3.236.190:45337)
0811 07:07:16.339569 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.339572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.339807 (+ 235us) server_negotiation.cc:407] Connection header received
0811 07:07:16.340849 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.340851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.340862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.340934 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.341799 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.342530 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.343551 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.343741 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.343811 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.346082 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.346083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.346083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.346091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.347578 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.348025 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.348026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.348026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.348079 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.348437 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.348438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.348438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.348560 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.348617 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.348879 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.348944 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:16.349216 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.339486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:45337 (local address 127.3.236.131:57985)
0811 07:07:16.339664 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:16.339707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:16.340749 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:16.340990 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:16.340993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:16.341369 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:16.341687 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.341695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.342635 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.342636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.343454 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:16.343457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.343826 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.343826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:16.343927 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.344532 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:16.344541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:16.345974 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:16.347683 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.347693 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.347695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.347949 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.348173 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:16.348174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:16.348175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:16.348370 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:16.348625 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:16.348631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:16.348756 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 07:07:16.348839 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:07:16.348971 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260811 07:07:16.349746 14543 heartbeater.cc:347] Connected to a master server at 127.3.236.190:45337
I20260811 07:07:16.349841 14543 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:16.350009 14543 heartbeater.cc:511] Master 127.3.236.190:45337 requested a full tablet report, sending...
I20260811 07:07:16.350375 13743 ts_manager.cc:194] Registered new tserver with Master: b6627fea0de54ec7aac319621edc7011 (127.3.236.131:40899)
I20260811 07:07:16.350711 13743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:57985
I20260811 07:07:16.353430 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/test-admin.keytab.
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.371292 4018 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 07:07:16.373194 14553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:16.372362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.3.236.129:45999)
0811 07:07:16.372534 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.372539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.372559 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:07:16.372587 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.372589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.372605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.372674 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.373078 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59156: BlockingRecv error: recv got EOF from 127.0.0.1:59156 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
W20260811 07:07:16.373207 4018 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc' of type FILE
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.374228 4018 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.375121 4018 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:07:16.383282 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.375674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.3.236.129:45999)
0811 07:07:16.375720 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.375722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.375759 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:07:16.375831 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.375832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.375841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.375889 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.376417 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.377159 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.377882 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.378201 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.378255 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.380535 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.380545 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.380545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.380552 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.382252 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.382597 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.382600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.382601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.382656 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.382816 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.382817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.382817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.382946 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.383061 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.383131 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.383197 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 07:07:16.385205 14553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:16.384676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.3.236.129:45999)
0811 07:07:16.384735 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.384737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.384777 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:07:16.384835 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.384835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.384844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.384886 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.385143 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59178: BlockingRecv error: recv got EOF from 127.0.0.1:59178 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 07:07:16.385291 4018 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc' of type FILE
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.386288 4018 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.387147 4018 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:07:16.393931 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.387685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.3.236.129:45999)
0811 07:07:16.387729 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.387730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.387741 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:16.387793 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.387793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.387800 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.387844 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.388197 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.388721 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.389352 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.389540 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.389588 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.391448 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.391449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.391449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.391455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.392846 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.393152 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.393152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.393153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.393203 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.393393 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.393394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.393394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.393481 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.393553 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.393723 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.393783 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 07:07:16.395956 14553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:16.395518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.3.236.129:45999)
0811 07:07:16.395554 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.395556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.395583 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:07:16.395636 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.395637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.395643 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.395686 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.395907 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59206: BlockingRecv error: recv got EOF from 127.0.0.1:59206 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 07:07:16.396054 4018 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786432006544353-4018-0/krb5kdc/krb5cc' of type FILE
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.397075 4018 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:16.398007 4018 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[13658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.129@KRBTEST.COM
I20260811 07:07:16.404908 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:16.398521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.3.236.129:45999)
0811 07:07:16.398581 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:16.398582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:16.398629 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:07:16.398676 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:16.398676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:16.398684 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:16.398729 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:16.399136 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.399578 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.400249 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:16.400484 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:16.400538 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:16.402535 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:16.402537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:16.402537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:16.402543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:16.403882 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.404195 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.404196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.404196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.404245 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:16.404468 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:16.404469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:16.404469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:16.404557 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:16.404629 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:16.404725 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 07:07:16.404779 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:07:16.405586 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 13855
I20260811 07:07:16.420838 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 14112
I20260811 07:07:16.441617 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 14333
I20260811 07:07:16.452126 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 13679
2026-08-11T07:07:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2210 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[14570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[14570](info): set up 2 sockets
Aug 11 07:07:16 dist-test-slave-2d19 krb5kdc[14570](info): commencing operation
krb5kdc: starting...
W20260811 07:07:17.733484 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:17 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab.
Entry for principal oryx/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab.
I20260811 07:07:17.863519 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42057
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab
--principal=oryx/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:17.946893 14597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:17.947154 14597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:17.947191 14597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:17.948345 14597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:17.948421 14597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:17.948443 14597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:17.948464 14597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:17.948494 14597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:17.950366 14597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab
--principal=oryx/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:17.950683 14597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:17.950940 14597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:17.956420 14617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:17.956431 14607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:17.956432 14609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:17.956593 14597 server_base.cc:1034] running on GCE node
I20260811 07:07:17.956825 14597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:17.957082 14597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:17.958238 14597 hybrid_clock.cc:648] HybridClock initialized: now 1786432037958222 us; error 34 us; skew 500 ppm
Aug 11 07:07:17 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:17.963985 14597 init.cc:377] Logged in from keytab as oryx/127.3.236.190@KRBTEST.COM (short username oryx)
I20260811 07:07:17.964509 14597 webserver.cc:460] Webserver started at http://127.3.236.190:37273/ using document root <none> and password file <none>
I20260811 07:07:17.964751 14597 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:17.964821 14597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:17.964978 14597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:17.966063 14597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "81e196cba1714bc5ad86f0de1abc2994"
format_stamp: "Formatted at 2026-08-11 07:07:17 on dist-test-slave-2d19"
server_key: "30ec0a21514c028536c5329bf4d401d8"
server_key_iv: "8d7545a1ceaa4fff0355ccb4616c3e55"
server_key_version: "encryptionkey@0"
I20260811 07:07:17.966454 14597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "81e196cba1714bc5ad86f0de1abc2994"
format_stamp: "Formatted at 2026-08-11 07:07:17 on dist-test-slave-2d19"
server_key: "30ec0a21514c028536c5329bf4d401d8"
server_key_iv: "8d7545a1ceaa4fff0355ccb4616c3e55"
server_key_version: "encryptionkey@0"
I20260811 07:07:17.967860 14597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:17.968719 14626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:17.968897 14597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:17.968989 14597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "81e196cba1714bc5ad86f0de1abc2994"
format_stamp: "Formatted at 2026-08-11 07:07:17 on dist-test-slave-2d19"
server_key: "30ec0a21514c028536c5329bf4d401d8"
server_key_iv: "8d7545a1ceaa4fff0355ccb4616c3e55"
server_key_version: "encryptionkey@0"
I20260811 07:07:17.969044 14597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:17.982928 14597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:17.985317 14597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:17.985435 14597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:17.990304 14597 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:42057
I20260811 07:07:17.990403 14711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:42057 every 8 connection(s)
I20260811 07:07:17.990778 14597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:17.991786 14713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:17.994251 14713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Bootstrap starting.
I20260811 07:07:17.994849 14713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:17.995095 14713 log.cc:824] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:17.995801 14713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: No bootstrap required, opened a new log
I20260811 07:07:17.997542 14713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } }
I20260811 07:07:17.997689 14713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:17.997713 14713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e196cba1714bc5ad86f0de1abc2994, State: Initialized, Role: FOLLOWER
I20260811 07:07:17.997855 14713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } }
I20260811 07:07:17.997943 14713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:17.997968 14713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:17.998005 14713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:17.998798 14713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } }
I20260811 07:07:17.998886 14713 leader_election.cc:302] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81e196cba1714bc5ad86f0de1abc2994; no voters:
I20260811 07:07:17.999070 14713 leader_election.cc:288] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:17.999109 14716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:17.999217 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 14597
I20260811 07:07:17.999328 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:17.999318 14716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 LEADER]: Becoming Leader. State: Replica: 81e196cba1714bc5ad86f0de1abc2994, State: Running, Role: LEADER
I20260811 07:07:17.999368 14713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:17.999421 14716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } }
I20260811 07:07:17.999557 4018 external_mini_cluster.cc:1428] Setting key 1ac6200b7b6628af1cef18b1defe2bf2
I20260811 07:07:17.999884 14723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e196cba1714bc5ad86f0de1abc2994. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } }
I20260811 07:07:18.000083 14723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:18.001245 14734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:18.001287 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } }
I20260811 07:07:18.001377 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:18.001781 14734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:18.003862 14734 catalog_manager.cc:1347] Generated new cluster ID: 18cc8b0644b14921b74dec4700c9a05f
I20260811 07:07:18.003927 14734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:18.011197 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.000355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.3.236.190:42057)
0811 07:07:18.001120 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.001129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.001164 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:07:18.001315 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.001325 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.001387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.001581 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.002289 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.003525 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.004483 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.004902 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.005034 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.007429 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.007434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.007434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.007448 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.009434 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.009856 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.009859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.009860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.009989 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.010205 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.010206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.010206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.010536 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.010720 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.010797 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.010957 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":588,"threads_started":1}
I20260811 07:07:18.020681 14734 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:18.021435 14734 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:18.033481 14734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Generated new TSK 0
I20260811 07:07:18.033710 14734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
I20260811 07:07:18.067755 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:18.153848 14752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:18.154089 14752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:18.154131 14752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:18.155295 14752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:18.155362 14752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:18.155411 14752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:18.156972 14752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:18.157234 14752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:18.157472 14752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:07:18.162815 14752 server_base.cc:1034] running on GCE node
W20260811 07:07:18.162820 14763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.162786 14764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.162791 14766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.163231 14752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:18.163506 14752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:18.164680 14752 hybrid_clock.cc:648] HybridClock initialized: now 1786432038164664 us; error 26 us; skew 500 ppm
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:18.167309 14752 init.cc:377] Logged in from keytab as oryx/127.3.236.129@KRBTEST.COM (short username oryx)
I20260811 07:07:18.167791 14752 webserver.cc:460] Webserver started at http://127.3.236.129:42445/ using document root <none> and password file <none>
I20260811 07:07:18.168073 14752 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.168150 14752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.168340 14752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:18.169406 14752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "15322815fc094adca693bd9b455d756e"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "7ebb7f01a21235052bcb55d45bafa5d3"
server_key_iv: "6ba8925ab0a7b0efe7fef0cb1e636f01"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.169773 14752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "15322815fc094adca693bd9b455d756e"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "7ebb7f01a21235052bcb55d45bafa5d3"
server_key_iv: "6ba8925ab0a7b0efe7fef0cb1e636f01"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.171126 14752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:18.172049 14781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.172290 14752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:18.172410 14752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "15322815fc094adca693bd9b455d756e"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "7ebb7f01a21235052bcb55d45bafa5d3"
server_key_iv: "6ba8925ab0a7b0efe7fef0cb1e636f01"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.172495 14752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.211326 14752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:18.213825 14752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:18.214005 14752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:18.214277 14752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:18.214627 14752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:18.214682 14752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.214723 14752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:18.214752 14752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.220829 14752 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:33919
I20260811 07:07:18.220880 14981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:33919 every 8 connection(s)
I20260811 07:07:18.221225 14752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:18.224472 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 14752
I20260811 07:07:18.224606 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:18.224815 4018 external_mini_cluster.cc:1428] Setting key 5491552b88381f2f01e17ffe71858ff9
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:18.232036 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.221950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:40023 (local address 127.3.236.190:42057)
0811 07:07:18.222041 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.222043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.222233 (+ 190us) server_negotiation.cc:407] Connection header received
0811 07:07:18.222723 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.222724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.222739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.222793 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.223752 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.224318 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.225767 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.225973 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.226034 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.228834 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.228836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.228836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.228852 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.230365 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.230877 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.230878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.230878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.230930 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.231272 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.231273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.231273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.231394 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.231471 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.231848 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.231946 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:07:18.232358 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.221905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:40023)
0811 07:07:18.222113 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.222151 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.222637 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.222848 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.222851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.223431 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.223647 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.223653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.224417 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.224418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.225684 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.225688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.226062 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.226063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.226217 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.226738 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.226749 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.228682 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.230447 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.230456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.230459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.230800 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.231027 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.231028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.231028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.231179 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.231474 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.231482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.231671 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.231808 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.232020 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260811 07:07:18.232993 14983 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:18.233099 14983 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:18.233321 14983 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:18.233942 14658 ts_manager.cc:194] Registered new tserver with Master: 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:18.234594 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:40023
WARNING: no policy specified for oryx/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
I20260811 07:07:18.255841 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:18.341288 14994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:18.341516 14994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:18.341542 14994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:18.342749 14994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:18.342789 14994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:18.342816 14994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:18.344288 14994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:18.344496 14994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:18.344682 14994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:18.350275 15002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.350284 15001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.350284 15004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.350533 14994 server_base.cc:1034] running on GCE node
I20260811 07:07:18.350802 14994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:18.351054 14994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:18.352195 14994 hybrid_clock.cc:648] HybridClock initialized: now 1786432038352173 us; error 39 us; skew 500 ppm
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:18.355015 14994 init.cc:377] Logged in from keytab as oryx/127.3.236.130@KRBTEST.COM (short username oryx)
I20260811 07:07:18.355521 14994 webserver.cc:460] Webserver started at http://127.3.236.130:46599/ using document root <none> and password file <none>
I20260811 07:07:18.355775 14994 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.355847 14994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.355998 14994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:18.357128 14994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "8ac1b62c4f374119a380b0f741347d63"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "cfa5ad85954e31d0c0900797b3c6082c"
server_key_iv: "7b0af9857f14842cf2483deaf2c49c70"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.357491 14994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "8ac1b62c4f374119a380b0f741347d63"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "cfa5ad85954e31d0c0900797b3c6082c"
server_key_iv: "7b0af9857f14842cf2483deaf2c49c70"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.358893 14994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:18.359680 15017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.359863 14994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:18.359969 14994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "8ac1b62c4f374119a380b0f741347d63"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "cfa5ad85954e31d0c0900797b3c6082c"
server_key_iv: "7b0af9857f14842cf2483deaf2c49c70"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.360054 14994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.372933 14994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:18.375545 14994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:18.375710 14994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:18.375973 14994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:18.376413 14994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:18.376469 14994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.376506 14994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:18.376551 14994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.382754 14994 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:35569
I20260811 07:07:18.382807 15223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:35569 every 8 connection(s)
I20260811 07:07:18.383137 14994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:18.391748 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 14994
I20260811 07:07:18.391872 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:18.392127 4018 external_mini_cluster.cc:1428] Setting key e58f87afbf641bfaeaba2dbd99ec2206
I20260811 07:07:18.392305 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.383887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:60477 (local address 127.3.236.190:42057)
0811 07:07:18.383955 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.383957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.384189 (+ 232us) server_negotiation.cc:407] Connection header received
0811 07:07:18.384706 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.384707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.384717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.384780 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.385781 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.386273 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.387251 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.387432 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.387481 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.389610 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.389612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.389612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.389619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.390816 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.391230 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.391231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.391231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.391280 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.391493 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.391493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.391494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.391601 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.391652 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.392079 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.392150 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:07:18.392508 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.383841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:60477)
0811 07:07:18.384053 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.384097 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.384615 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.384860 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.384864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.385484 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.385688 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.385693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.386362 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.386362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.387170 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.387172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.387504 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.387505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.387616 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.388094 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.388103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.389520 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.390907 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.390914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.390916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.391164 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.391337 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.391338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.391338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.391438 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.391668 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.391677 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.391856 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.392050 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.392233 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260811 07:07:18.393038 15224 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:18.393151 15224 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:18.393354 15224 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:18.393715 14658 ts_manager.cc:194] Registered new tserver with Master: 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130:35569)
I20260811 07:07:18.394079 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:60477
WARNING: no policy specified for oryx/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
I20260811 07:07:18.421483 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:18.507601 15237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:18.507846 15237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:18.507890 15237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:18.509153 15237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:18.509224 15237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:18.509269 15237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:18.510782 15237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:18.511058 15237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:18.511297 15237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:18.516588 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.516583 15250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.516614 15237 server_base.cc:1034] running on GCE node
W20260811 07:07:18.516584 15247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.517069 15237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:18.517278 15237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:18.518431 15237 hybrid_clock.cc:648] HybridClock initialized: now 1786432038518412 us; error 30 us; skew 500 ppm
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:18.521034 15237 init.cc:377] Logged in from keytab as oryx/127.3.236.131@KRBTEST.COM (short username oryx)
I20260811 07:07:18.521481 15237 webserver.cc:460] Webserver started at http://127.3.236.131:37143/ using document root <none> and password file <none>
I20260811 07:07:18.521733 15237 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.521804 15237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.521958 15237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:18.523041 15237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "41758d2ff486493987ab4e49efa9b070"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "68bcbf2e01187610746ae3af28f0b666"
server_key_iv: "d90b7967b84ec71f30d3613349a2e92d"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.523439 15237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "41758d2ff486493987ab4e49efa9b070"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "68bcbf2e01187610746ae3af28f0b666"
server_key_iv: "d90b7967b84ec71f30d3613349a2e92d"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.524920 15237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:18.525708 15264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.525887 15237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:18.526000 15237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "41758d2ff486493987ab4e49efa9b070"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "68bcbf2e01187610746ae3af28f0b666"
server_key_iv: "d90b7967b84ec71f30d3613349a2e92d"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.526081 15237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.544749 15237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:18.547258 15237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:18.547435 15237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:18.547711 15237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:18.548044 15237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:18.548110 15237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.548156 15237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:18.548197 15237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.554495 15237 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:46375
I20260811 07:07:18.554543 15464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:46375 every 8 connection(s)
I20260811 07:07:18.554917 15237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:18.557721 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 15237
I20260811 07:07:18.557816 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:18.557976 4018 external_mini_cluster.cc:1428] Setting key 429695042b325c3a5e40c98502da9c4c
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:18.564810 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.555730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:48375 (local address 127.3.236.190:42057)
0811 07:07:18.555815 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.555818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.555989 (+ 171us) server_negotiation.cc:407] Connection header received
0811 07:07:18.556556 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.556557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.556568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.556641 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.557584 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.558285 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.559600 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.559788 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.559844 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.562135 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.562136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.562137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.562144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.563415 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.563784 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.563784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.563785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.563835 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.564151 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.564152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.564152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.564351 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.564420 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.564672 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.564743 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:07:18.565011 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.555686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:48375)
0811 07:07:18.555854 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.555897 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.556463 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.556697 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.556700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.557292 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.557492 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.557497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.558403 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.558405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.559516 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.559519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.559865 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.559865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.559969 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.560648 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.560657 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.562034 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.563489 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.563496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.563498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.563716 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.563894 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.563895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.563895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.564048 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.564415 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.564421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.564541 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.564633 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.564761 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260811 07:07:18.565622 15465 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:18.565737 15465 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:18.565944 15465 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:18.566357 14658 ts_manager.cc:194] Registered new tserver with Master: 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375)
I20260811 07:07:18.566702 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:48375
I20260811 07:07:18.570006 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.189.keytab.
Entry for principal oryx/127.3.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.189.keytab.
I20260811 07:07:18.599164 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:35729
--webserver_interface=127.3.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.189.keytab
--principal=oryx/127.3.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42057,127.3.236.189:35729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:18.684309 15472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:18.684566 15472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:18.684610 15472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:18.685806 15472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:18.685875 15472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:18.685899 15472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:18.685920 15472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:18.685952 15472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:18.687500 15472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42057,127.3.236.189:35729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.189.keytab
--principal=oryx/127.3.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:35729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
W20260811 07:07:18.687662 15472 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.236.190:42057,127.3.236.189:35729'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 07:07:18.687830 15472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:18.688076 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:18.693046 15481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.693070 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.693136 15478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.693456 15472 server_base.cc:1034] running on GCE node
I20260811 07:07:18.693686 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:18.693933 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:18.695082 15472 hybrid_clock.cc:648] HybridClock initialized: now 1786432038695069 us; error 28 us; skew 500 ppm
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:18.697927 15472 init.cc:377] Logged in from keytab as oryx/127.3.236.189@KRBTEST.COM (short username oryx)
I20260811 07:07:18.698455 15472 webserver.cc:460] Webserver started at http://127.3.236.189:34993/ using document root <none> and password file <none>
I20260811 07:07:18.698696 15472 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.698768 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.698927 15472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:18.700209 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/instance:
uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "15525da7bd3ec93dc442226c68b3a2ec"
server_key_iv: "f29482d64d568fbb32880a3454b5a0b7"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.700702 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal/instance:
uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "15525da7bd3ec93dc442226c68b3a2ec"
server_key_iv: "f29482d64d568fbb32880a3454b5a0b7"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.702107 15472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:18.702888 15495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.703063 15472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:18.703140 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "15525da7bd3ec93dc442226c68b3a2ec"
server_key_iv: "f29482d64d568fbb32880a3454b5a0b7"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.703193 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.740008 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:18.742067 15472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:18.742254 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:18.747382 15472 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.189:35729
I20260811 07:07:18.747453 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.189:35729 every 8 connection(s)
I20260811 07:07:18.747822 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
I20260811 07:07:18.748999 15586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:18.750799 15586 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:07:18.755012 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 15472
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.189@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:18.761307 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.751520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.3.236.190:42057)
0811 07:07:18.751583 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.751585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.751825 (+ 240us) server_negotiation.cc:407] Connection header received
0811 07:07:18.752441 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.752442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.752453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.752534 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.753226 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.753832 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.754827 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.755132 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.755208 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.758049 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.758051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.758052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.758059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.759732 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.760194 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.760196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.760196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.760277 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.760550 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.760552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.760552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.760761 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.760836 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.761107 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.761192 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:07:18.761466 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.751465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40374)
0811 07:07:18.751686 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.751732 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.752335 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.752609 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.752612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.752870 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.753131 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.753135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.753913 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.753914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.754716 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.754718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.755248 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.755249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.755352 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.755994 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.756007 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.757953 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.759856 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.759863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.759866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.760110 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.760337 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.760344 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.760344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.760457 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.760841 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.760848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.760967 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.761058 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.761189 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260811 07:07:18.762213 15586 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:07:18.764832 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.755716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.3.236.189:35729)
0811 07:07:18.755934 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.755939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.756131 (+ 192us) server_negotiation.cc:407] Connection header received
0811 07:07:18.756153 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.756155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.756175 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.756289 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.757128 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.758088 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.758846 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.759166 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.759248 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.761714 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.761723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.761723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.761730 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.763535 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.763910 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.763913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.763913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.763982 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.764188 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.764189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.764190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.764384 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.764567 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.764639 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.764721 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":142,"threads_started":1}
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.189@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:18.769665 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.762535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37690)
0811 07:07:18.762577 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.762586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.762619 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.763151 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.763152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.763300 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.763373 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.763374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.764063 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.764064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.764669 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.764671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.765143 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.765144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.765241 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.765649 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.765654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.766811 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.768745 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.768746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.768746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.768913 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.769149 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.769150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.769150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.769212 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.769428 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.769429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.769464 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.769523 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.769605 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 07:07:18.769825 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.762845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.3.236.189:35729)
0811 07:07:18.763032 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.763034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.763039 (+ 5us) server_negotiation.cc:407] Connection header received
0811 07:07:18.763054 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.763054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.763061 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.763140 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.763526 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.763982 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.764766 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.765072 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.765141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.766881 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.766882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.766883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.766888 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.768651 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.768984 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.768985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.768985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.769036 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.769276 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.769277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.769277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.769369 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.769418 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.769540 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.769650 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":170,"thread_start_us":124,"threads_started":1}
I20260811 07:07:18.771451 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Bootstrap starting.
I20260811 07:07:18.772306 15586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:18.772651 15586 log.cc:824] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:18.773396 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: No bootstrap required, opened a new log
I20260811 07:07:18.775113 15586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } }
I20260811 07:07:18.775240 15586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:18.775261 15586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a83fc5015d4b17b0e7dcea87b3805f, State: Initialized, Role: FOLLOWER
I20260811 07:07:18.775437 15586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } }
I20260811 07:07:18.775707 15599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } }
I20260811 07:07:18.775792 15586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:18.775803 15599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:07:18.777935 15625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:07:18.778038 15625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:07:18.779798 15472 master_runner.cc:428] Detected that this master 74a83fc5015d4b17b0e7dcea87b3805f is joining an existing cluster
I20260811 07:07:18.779902 15472 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.236.189:35729
I20260811 07:07:18.788421 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.781091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.3.236.190:42057)
0811 07:07:18.781141 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.781143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.781297 (+ 154us) server_negotiation.cc:407] Connection header received
0811 07:07:18.781375 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.781376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.781386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.781443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.781977 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.782488 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.783402 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.783693 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.783733 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.785081 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.785083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.785083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.785092 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.786977 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787483 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.787484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.787484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.787542 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787787 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.787788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.787788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.787931 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.787990 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.788212 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.788321 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:07:18.788499 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.781494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.3.236.189:35729)
0811 07:07:18.781532 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.781533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.781541 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:18.781561 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.781562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.781570 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.781631 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.782071 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.782853 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.783820 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.784147 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.784241 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.785586 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.785587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.785587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.785600 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.787285 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787662 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.787664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.787664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.787713 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787930 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.787931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.787931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.788041 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.788130 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.788304 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.788378 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:07:18.788443 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.781057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40384)
0811 07:07:18.781229 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.781238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.781313 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.781494 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.781495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.781758 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.781845 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.781847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.782563 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.782564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.783309 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.783311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.783834 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.783835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.783886 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.784536 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.784541 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.784970 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.787079 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.787081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.787403 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.787622 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.787623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.787694 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.788042 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.788043 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.788102 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.788176 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.788317 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260811 07:07:18.788805 15638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.781317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37696)
0811 07:07:18.781450 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.781458 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.781503 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.781644 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.781645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.781842 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.781962 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.781964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.782944 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.782945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.783732 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.783734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.784275 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.784277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.784365 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.785135 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.785140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.785471 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.787360 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.787361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.787609 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.787791 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.787792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.787793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.787866 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.788125 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.788126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.788166 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.788461 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.788628 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.190@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:18.798290 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.790839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37708)
0811 07:07:18.790972 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.791013 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.791089 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.791228 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.791230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.791466 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.791516 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.791517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.792481 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.792482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.793051 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.793052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.793519 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.793520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.793585 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.794032 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.794036 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.795587 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.797251 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.797257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.797259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.797451 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.797655 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.797656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.797656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.797709 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.798000 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.798006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.798047 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.798118 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.798207 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260811 07:07:18.798338 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.790900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.3.236.189:35729)
0811 07:07:18.790935 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.790937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.791083 (+ 146us) server_negotiation.cc:407] Connection header received
0811 07:07:18.791148 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.791149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.791159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.791225 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.791584 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.792408 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.793130 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.793442 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.793514 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.795677 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.795678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.795679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.795686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.797167 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.797526 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.797527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.797528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.797588 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.797784 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.797785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.797785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.797917 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.798006 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.798166 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.798246 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:07:18.798938 14658 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.236.189:35729: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
member_type: NON_VOTER
last_known_addr {
host: "127.3.236.189"
port: 35729
}
attrs {
promote: true
}
}
dest_uuid: "81e196cba1714bc5ad86f0de1abc2994"
cas_config_opid_index: -1
I20260811 07:07:18.799305 14658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } }
I20260811 07:07:18.799873 15643 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:18.800364 15539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:18.801054 15643 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.236.189:35729
I20260811 07:07:18.801322 15472 master.cc:537] Master@127.3.236.189:35729 shutting down...
I20260811 07:07:18.801430 15539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 07:07:18.801803 15642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:18.801975 15642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:18.801812 15643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:07:18.802793 14633 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 07:07:18.804337 14633 consensus_peers.cc:487] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 -> Peer 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189:35729): Couldn't send request to peer 74a83fc5015d4b17b0e7dcea87b3805f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 07:07:18.804845 15472 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:18.804965 15472 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:18.805022 15472 tablet_replica.cc:330] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: stopping tablet replica
I20260811 07:07:18.817950 15472 master.cc:552] Master@127.3.236.189:35729 shutdown complete.
I20260811 07:07:18.818028 15472 master_runner.cc:305] Clearing existing system tablet
I20260811 07:07:18.818166 15472 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.818233 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.818854 15472 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:18.819387 15651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.819566 15472 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.819626 15472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.819672 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "15525da7bd3ec93dc442226c68b3a2ec"
server_key_iv: "f29482d64d568fbb32880a3454b5a0b7"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.820472 15472 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:18.822229 15472 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 07:07:18.822290 15472 log.cc:1196] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 07:07:18.822490 15472 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Deleting consensus metadata
I20260811 07:07:18.822629 15472 master_runner.cc:315] Copying system tablet from 127.3.236.190:42057
I20260811 07:07:18.823284 15472 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: tablet copy: Beginning tablet copy session from remote peer at address 127.3.236.190:42057
I20260811 07:07:18.830376 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.823621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40392)
0811 07:07:18.823766 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:18.823780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:18.823867 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:18.824063 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:18.824064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:18.824241 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:18.824365 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.824367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.825326 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.825327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.825983 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:18.825985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.826470 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.826471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:18.826524 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.827041 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:18.827047 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:18.827366 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:18.829319 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.829320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.829320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.829489 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.829706 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:18.829706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:18.829707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:18.829758 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:18.830093 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:18.830094 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:18.830138 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 07:07:18.830210 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:07:18.830295 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260811 07:07:18.830435 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.823732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.3.236.190:42057)
0811 07:07:18.823807 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.823810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.823850 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:07:18.823974 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.823975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.823986 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.824057 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.824441 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.825229 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.826078 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.826384 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.826464 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.827476 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.827478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.827478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.827486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.829230 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.829566 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.829567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.829567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.829635 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.829847 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.829848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.829848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.829993 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.830069 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.830256 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.830340 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:07:18.831010 14693 tablet_copy_service.cc:140] P 81e196cba1714bc5ad86f0de1abc2994: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 74a83fc5015d4b17b0e7dcea87b3805f ({username='oryx', principal='oryx/127.3.236.189@KRBTEST.COM'} at 127.0.0.1:40392)
I20260811 07:07:18.831115 14693 tablet_copy_service.cc:161] P 81e196cba1714bc5ad86f0de1abc2994: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 74a83fc5015d4b17b0e7dcea87b3805f at {username='oryx', principal='oryx/127.3.236.189@KRBTEST.COM'} at 127.0.0.1:40392: session id = 74a83fc5015d4b17b0e7dcea87b3805f-00000000000000000000000000000000
I20260811 07:07:18.832044 14693 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 07:07:18.832746 15472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 07:07:18.833667 15472 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: tablet copy: Starting download of 0 data blocks...
I20260811 07:07:18.833819 15472 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: tablet copy: Starting download of 1 WAL segments...
I20260811 07:07:18.834793 15472 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 07:07:18.835665 14693 tablet_copy_service.cc:342] P 81e196cba1714bc5ad86f0de1abc2994: Request end of tablet copy session 74a83fc5015d4b17b0e7dcea87b3805f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.236.189@KRBTEST.COM'} at 127.0.0.1:40392
I20260811 07:07:18.835729 14693 tablet_copy_service.cc:433] P 81e196cba1714bc5ad86f0de1abc2994: ending tablet copy session 74a83fc5015d4b17b0e7dcea87b3805f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 74a83fc5015d4b17b0e7dcea87b3805f
I20260811 07:07:18.838418 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:18.839591 15661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.839632 15662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:18.839756 15666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:18.839890 15472 server_base.cc:1034] running on GCE node
I20260811 07:07:18.840224 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:18.840400 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:18.841553 15472 hybrid_clock.cc:648] HybridClock initialized: now 1786432038841545 us; error 27 us; skew 500 ppm
Aug 11 07:07:18 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432038, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:18.843045 15472 init.cc:377] Logged in from keytab as oryx/127.3.236.189@KRBTEST.COM (short username oryx)
I20260811 07:07:18.843349 15472 webserver.cc:460] Webserver started at http://127.3.236.189:35593/ using document root <none> and password file <none>
I20260811 07:07:18.843477 15472 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:18.843531 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:18.844152 15472 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:18.844726 15675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.844888 15472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:18.844951 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "15525da7bd3ec93dc442226c68b3a2ec"
server_key_iv: "f29482d64d568fbb32880a3454b5a0b7"
server_key_version: "encryptionkey@0"
I20260811 07:07:18.845011 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:18.864070 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:18.865758 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:18.869323 15472 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.189:35729
I20260811 07:07:18.869618 15771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.189:35729 every 8 connection(s)
I20260811 07:07:18.869837 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
I20260811 07:07:18.870806 15772 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:07:18.871368 15772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Bootstrap starting.
I20260811 07:07:18.874681 15772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:07:18.874955 15772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: Bootstrap complete.
I20260811 07:07:18.875190 15772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } }
I20260811 07:07:18.875291 15772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74a83fc5015d4b17b0e7dcea87b3805f, State: Initialized, Role: LEARNER
I20260811 07:07:18.875334 15772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } }
I20260811 07:07:18.875543 15772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:18.875560 15776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:18.875727 15776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: LEARNER
I20260811 07:07:18.877434 15800 catalog_manager.cc:1259] Loaded cluster ID: 18cc8b0644b14921b74dec4700c9a05f
I20260811 07:07:18.877530 15800 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: loading cluster ID for follower catalog manager: success
I20260811 07:07:18.878623 15800 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: acquiring CA information for follower catalog manager: success
I20260811 07:07:18.878913 15800 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:07:18.923547 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:18.916995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.3.236.189:35729)
0811 07:07:18.917159 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:18.917160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:18.917169 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:18.917193 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:18.917194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:18.917203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:18.917280 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:18.917845 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.918494 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.919258 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:18.919462 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:18.919542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:18.920872 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:18.920878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:18.920879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:18.920886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:18.922333 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.922696 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.922697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.922698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.922762 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:18.922967 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:18.922968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:18.922968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:18.923084 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:18.923196 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:18.923310 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 07:07:18.923398 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.3.236.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.188.keytab.
Entry for principal oryx/127.3.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.188.keytab.
I20260811 07:07:18.955464 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:44219
--webserver_interface=127.3.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.188.keytab
--principal=oryx/127.3.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:19.048841 15805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:19.049098 15805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:19.049141 15805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:19.050410 15805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:19.050489 15805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:19.050516 15805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:19.050550 15805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:19.050580 15805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:19.052136 15805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.188.keytab
--principal=oryx/127.3.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:19.052441 15805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:19.052691 15805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:19.057780 15814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.057785 15813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.057915 15805 server_base.cc:1034] running on GCE node
W20260811 07:07:19.057760 15817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.058290 15805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:19.058619 15805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:19.059792 15805 hybrid_clock.cc:648] HybridClock initialized: now 1786432039059766 us; error 37 us; skew 500 ppm
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:19.062398 15805 init.cc:377] Logged in from keytab as oryx/127.3.236.188@KRBTEST.COM (short username oryx)
I20260811 07:07:19.062913 15805 webserver.cc:460] Webserver started at http://127.3.236.188:34197/ using document root <none> and password file <none>
I20260811 07:07:19.063148 15805 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.063220 15805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.063354 15805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:19.064479 15805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/instance:
uuid: "38490b4a056c406a939544918a5b6aa9"
format_stamp: "Formatted at 2026-08-11 07:07:19 on dist-test-slave-2d19"
server_key: "ee0179ca23e1944d37a92ab5fab3eafa"
server_key_iv: "70ac65dc181c334d0870b817f615f446"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.064800 15805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal/instance:
uuid: "38490b4a056c406a939544918a5b6aa9"
format_stamp: "Formatted at 2026-08-11 07:07:19 on dist-test-slave-2d19"
server_key: "ee0179ca23e1944d37a92ab5fab3eafa"
server_key_iv: "70ac65dc181c334d0870b817f615f446"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.066221 15805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:19.066933 15833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.067107 15805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:19.067217 15805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "38490b4a056c406a939544918a5b6aa9"
format_stamp: "Formatted at 2026-08-11 07:07:19 on dist-test-slave-2d19"
server_key: "ee0179ca23e1944d37a92ab5fab3eafa"
server_key_iv: "70ac65dc181c334d0870b817f615f446"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.067293 15805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.076280 15805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:19.078606 15805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:19.078760 15805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:19.083554 15805 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.188:44219
I20260811 07:07:19.083587 15922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.188:44219 every 8 connection(s)
I20260811 07:07:19.083923 15805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
I20260811 07:07:19.084966 15923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:19.086649 15923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:07:19.091151 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 15805
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.188@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:19.096311 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.087424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.3.236.190:42057)
0811 07:07:19.087495 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.087497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.087655 (+ 158us) server_negotiation.cc:407] Connection header received
0811 07:07:19.088213 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.088214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.088222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.088326 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.088948 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.089430 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.090616 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.090803 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.090860 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.093237 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.093238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.093239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.093246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.094777 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.095239 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.095241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.095241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.095311 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.095577 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.095579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.095579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.095702 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.095779 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.096070 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.096161 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:19.096457 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.087304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40398)
0811 07:07:19.087517 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.087558 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.088125 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.088402 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.088405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.088652 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.088873 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.088877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.089515 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.089516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.090524 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.090527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.090941 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.090942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.091038 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.091597 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.091608 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.093148 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.094856 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.094864 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.094866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.095130 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.095384 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.095385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.095385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.095502 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.095782 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.095790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.095923 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.096027 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.096179 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":129,"threads_started":1}
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
I20260811 07:07:19.097246 15923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:07:19.100872 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.091740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.3.236.188:44219)
0811 07:07:19.091903 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.091908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.091986 (+ 78us) server_negotiation.cc:407] Connection header received
0811 07:07:19.092052 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.092055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.092078 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.092180 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.092784 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.093764 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.094684 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.095074 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.095167 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.097813 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.097822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.097822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.097829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.099401 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.099783 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.099786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.099787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.099847 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.100044 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.100045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.100046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.100243 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.100602 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.100656 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.100767 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.188@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:19.104568 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.097519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37756)
0811 07:07:19.097567 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.097577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.097626 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.097823 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.097825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.097974 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.098015 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.098017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.099010 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.099011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.099582 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.099585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.099972 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.099973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.100070 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.100598 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.100603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.101978 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.103513 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.103513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.103514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.103671 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.103967 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.103968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.103968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.104028 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.104352 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.104353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.104386 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.104441 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.104511 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:07:19.104663 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.097619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.3.236.189:35729)
0811 07:07:19.097695 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.097697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.097705 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:19.097729 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.097730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.097739 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.097806 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.098131 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.098927 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.099674 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.099896 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.099975 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.102068 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.102070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.102070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.102078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.103413 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.103747 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.103748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.103748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.103804 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.104123 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.104124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.104124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.104246 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.104365 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.104477 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.104556 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:19.105145 15923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.188@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
I20260811 07:07:19.111006 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.105387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.0.0.1:49798)
0811 07:07:19.105447 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.105457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.105496 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.105612 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.105612 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.105715 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.105741 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.105742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.106314 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.106315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.106780 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.106781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.107084 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.107084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.107119 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.107529 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.107534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.108747 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.110054 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.110055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.110055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.110216 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.110414 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.110415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.110415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.110468 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.110705 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.110706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.110742 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.110810 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.110902 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:07:19.111060 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.105480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.3.236.188:44219)
0811 07:07:19.105512 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.105514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.105522 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:19.105539 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.105540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.105549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.105608 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.105816 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.106253 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.106845 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.107022 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.107069 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.108833 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.108834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.108834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.108841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.109953 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.110297 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.110298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.110298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.110350 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.110539 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.110540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.110541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.110639 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.110713 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.110835 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.110951 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":16}
I20260811 07:07:19.112582 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Bootstrap starting.
I20260811 07:07:19.113391 15923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:19.113680 15923 log.cc:824] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:19.114393 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: No bootstrap required, opened a new log
I20260811 07:07:19.115864 15923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } }
I20260811 07:07:19.115984 15923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:19.116006 15923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38490b4a056c406a939544918a5b6aa9, State: Initialized, Role: FOLLOWER
I20260811 07:07:19.116091 15923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } }
I20260811 07:07:19.116444 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } } }
I20260811 07:07:19.116508 15923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:19.116561 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:07:19.118221 15963 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:07:19.118285 15963 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:07:19.120535 15805 master_runner.cc:428] Detected that this master 38490b4a056c406a939544918a5b6aa9 is joining an existing cluster
I20260811 07:07:19.120586 15805 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.236.188:44219
I20260811 07:07:19.129089 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.122294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.3.236.188:44219)
0811 07:07:19.122337 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.122339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.122614 (+ 275us) server_negotiation.cc:407] Connection header received
0811 07:07:19.122696 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.122697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.122704 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.122730 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.123209 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.123879 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125059 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125334 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125371 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.126489 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.126490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.126490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.126497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.127902 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.128285 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.128286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.128287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.128341 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.128552 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.128553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.128553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.128661 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.128735 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.128884 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.129012 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":21}
I20260811 07:07:19.129112 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.122345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.0.0.1:49810)
0811 07:07:19.122555 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.122565 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.122635 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.122780 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.122781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.123020 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.123123 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.123125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.123939 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.123940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.124965 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.124968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125479 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.125548 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.126027 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.126032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.126377 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.128011 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.128012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.128012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.128190 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.128416 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.128417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.128418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.128481 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.128727 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.128728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.128789 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.128848 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.128953 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":32,"thread_start_us":85,"threads_started":1}
I20260811 07:07:19.153069 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 14752
I20260811 07:07:19.172183 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.121832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.3.236.189:35729)
0811 07:07:19.121878 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.121880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.122246 (+ 366us) server_negotiation.cc:407] Connection header received
0811 07:07:19.122371 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.122372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.122385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.122419 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.122916 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.123734 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125086 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125390 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125429 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.168113 (+ 42684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.168116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.168117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.168127 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.170694 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171156 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.171158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.171158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.171225 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171466 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.171468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.171468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.171638 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.171689 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.171937 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.172035 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:07:19.172335 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.121686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.3.236.190:42057)
0811 07:07:19.121741 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.121743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.122576 (+ 833us) server_negotiation.cc:407] Connection header received
0811 07:07:19.122680 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.122681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.122692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.122724 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.123241 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.123975 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125147 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125471 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125523 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.169105 (+ 43582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.169107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.169107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.169118 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.170854 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171362 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.171364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.171364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.171425 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171644 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.171645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.171646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.171780 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.171825 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.172083 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.172171 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:19.172638 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.121618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37770)
0811 07:07:19.122185 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.122197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.122291 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.122494 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.122495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.122739 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.122823 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.122826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.123826 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.123826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.125001 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.125002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125508 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.125563 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.153001 (+ 27438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.153007 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.167963 (+ 14956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.170805 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.170807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.170807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.171061 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.171304 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.171305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.171375 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.171744 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.171745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.171801 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.172357 (+ 556us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.172534 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":66,"thread_start_us":93,"threads_started":1}
I20260811 07:07:19.172771 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.121622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40408)
0811 07:07:19.122509 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.122519 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.122604 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.122785 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.122785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.123064 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.123139 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.123141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.124104 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.124104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.125032 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.125035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.125603 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.125604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.125668 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.152920 (+ 27252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.152926 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.167363 (+ 14437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.171005 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.171007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.171274 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.171503 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.171503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.171504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.171571 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.171873 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.171874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.171962 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.172452 (+ 490us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.172610 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":29,"thread_start_us":511,"threads_started":1}
I20260811 07:07:19.175182 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:33919
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:42057
--webserver_port=42445
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432037, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.190@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
I20260811 07:07:19.183593 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.175105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.3.236.188:44219)
0811 07:07:19.175144 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.175146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.175155 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:19.175227 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.175228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.175238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.175294 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.175676 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.176435 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.177506 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.177805 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.177867 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.180444 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.180446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.180446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.180454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.182224 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.182679 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.182681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.182681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.182747 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.182991 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.182992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.182992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.183139 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.183210 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.183401 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.183489 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:07:19.183646 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.175052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.0.0.1:49812)
0811 07:07:19.175108 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.175119 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.175157 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.175351 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.175352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.175529 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.175582 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.175584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.176525 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.176527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.177401 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.177404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.177925 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.177927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.178009 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.178476 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.178482 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.180222 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.182325 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.182326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.182327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.182587 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.182829 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.182829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.182830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.182907 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.183217 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.183218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.183276 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.183361 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.183496 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 07:07:19.184185 14658 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.236.188:44219: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "38490b4a056c406a939544918a5b6aa9"
member_type: NON_VOTER
last_known_addr {
host: "127.3.236.188"
port: 44219
}
attrs {
promote: true
}
}
dest_uuid: "81e196cba1714bc5ad86f0de1abc2994"
cas_config_opid_index: 5
I20260811 07:07:19.184513 14658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
I20260811 07:07:19.185065 15643 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 38490b4a056c406a939544918a5b6aa9 (127.3.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.186163 15643 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.236.188:44219
I20260811 07:07:19.186239 15643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.186304 15643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.186750 15881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:19.188442 15881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 07:07:19.188858 15642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:19.189334 15805 master.cc:537] Master@127.3.236.188:44219 shutting down...
I20260811 07:07:19.189949 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.185253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37776)
0811 07:07:19.185555 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.185567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.185620 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.185876 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.185877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.185879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.185946 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.185948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.187113 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.187114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.188522 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.188526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.189678 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.189679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.189748 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.189772 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.189821 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.189865 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260811 07:07:19.190462 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.185470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.3.236.189:35729)
0811 07:07:19.185520 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.185522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.185676 (+ 154us) server_negotiation.cc:407] Connection header received
0811 07:07:19.185707 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.185708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.185740 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.185806 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.186042 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.186862 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.188705 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.189400 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.189475 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.189558 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.190350 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.190397 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:07:19.190670 15722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 LEARNER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 07:07:19.191097 15642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 07:07:19.191699 15722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 38490b4a056c406a939544918a5b6aa9 (127.3.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.193284 15881 raft_consensus.cc:858] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
W20260811 07:07:19.193365 15881 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
I20260811 07:07:19.193470 15776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e196cba1714bc5ad86f0de1abc2994. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.193552 15776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: LEARNER
I20260811 07:07:19.193588 15776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.193625 15776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: LEARNER
I20260811 07:07:19.193805 15881 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:19.194396 15642 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: attempting to promote NON_VOTER 74a83fc5015d4b17b0e7dcea87b3805f to VOTER
I20260811 07:07:19.194563 15642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
I20260811 07:07:19.195057 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e196cba1714bc5ad86f0de1abc2994. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:19.195120 15722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 LEARNER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 07:07:19.195143 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 07:07:19.195179 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:19.195221 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 07:07:19.195247 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: NON_VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: true } } }
I20260811 07:07:19.195284 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 07:07:19.195524 15642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 07:07:19.195751 14629 consensus_peers.cc:487] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 -> Peer 38490b4a056c406a939544918a5b6aa9 (127.3.236.188:44219): Couldn't send request to peer 38490b4a056c406a939544918a5b6aa9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 07:07:19.196743 15977 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.196870 15979 mvcc.cc:205] Tried to move back new op lower bound from 7317225627783057408 to 7317225627646513152. Current Snapshot: MvccSnapshot[applied={T|T < 7317225627660587008}]
I20260811 07:07:19.197856 15643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.197948 15643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.198175 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.198318 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.198315 15722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.199208 15776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.199296 15776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:07:19.200484 15805 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 07:07:19.200593 15805 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 07:07:19.200626 15805 tablet_replica.cc:330] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: stopping tablet replica
I20260811 07:07:19.215850 15805 master.cc:552] Master@127.3.236.188:44219 shutdown complete.
I20260811 07:07:19.215967 15805 master_runner.cc:305] Clearing existing system tablet
I20260811 07:07:19.216111 15805 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.216169 15805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.217069 15805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:19.217666 15995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.217792 15805 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.217839 15805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:19.217862 15805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "38490b4a056c406a939544918a5b6aa9"
format_stamp: "Formatted at 2026-08-11 07:07:19 on dist-test-slave-2d19"
server_key: "ee0179ca23e1944d37a92ab5fab3eafa"
server_key_iv: "70ac65dc181c334d0870b817f615f446"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.218743 15805 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:19.220337 15805 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 07:07:19.220412 15805 log.cc:1196] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 07:07:19.220683 15805 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Deleting consensus metadata
I20260811 07:07:19.220824 15805 master_runner.cc:315] Copying system tablet from 127.3.236.190:42057
I20260811 07:07:19.221488 15805 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: tablet copy: Beginning tablet copy session from remote peer at address 127.3.236.190:42057
I20260811 07:07:19.229313 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.221924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.3.236.190:42057)
0811 07:07:19.221981 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.221984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.222097 (+ 113us) server_negotiation.cc:407] Connection header received
0811 07:07:19.222174 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.222175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.222189 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.222255 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.222643 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.223434 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.224570 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.224901 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.224996 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.226206 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.226208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.226208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.226217 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.227998 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.228463 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.228464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.228465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.228530 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.228755 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.228756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.228757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.228894 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.228963 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.229133 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.229217 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:07:19.229342 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.221812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40412)
0811 07:07:19.222001 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.222014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.222100 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.222275 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.222276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.222491 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.222552 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.222555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.223538 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.223540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.224470 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.224473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.225014 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.225016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.225089 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.225641 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.225646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.226101 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.228100 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.228101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.228102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.228371 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.228613 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.228614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.228614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.228689 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.228976 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.228977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.229030 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.229108 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.229223 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260811 07:07:19.234426 14693 tablet_copy_service.cc:140] P 81e196cba1714bc5ad86f0de1abc2994: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 38490b4a056c406a939544918a5b6aa9 ({username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:40412)
I20260811 07:07:19.234510 14693 tablet_copy_service.cc:161] P 81e196cba1714bc5ad86f0de1abc2994: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 38490b4a056c406a939544918a5b6aa9 at {username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:40412: session id = 38490b4a056c406a939544918a5b6aa9-00000000000000000000000000000000
I20260811 07:07:19.235221 14693 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 07:07:19.235785 15805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 07:07:19.237293 15805 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: tablet copy: Starting download of 0 data blocks...
I20260811 07:07:19.237445 15805 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: tablet copy: Starting download of 1 WAL segments...
I20260811 07:07:19.238180 15805 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 07:07:19.239048 14693 tablet_copy_service.cc:342] P 81e196cba1714bc5ad86f0de1abc2994: Request end of tablet copy session 38490b4a056c406a939544918a5b6aa9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:40412
I20260811 07:07:19.239099 14693 tablet_copy_service.cc:433] P 81e196cba1714bc5ad86f0de1abc2994: ending tablet copy session 38490b4a056c406a939544918a5b6aa9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 38490b4a056c406a939544918a5b6aa9
I20260811 07:07:19.242015 15805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:19.243171 16008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.245167 16009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.248418 16016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.249140 15805 server_base.cc:1034] running on GCE node
I20260811 07:07:19.249255 15805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:19.249404 15805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:19.250545 15805 hybrid_clock.cc:648] HybridClock initialized: now 1786432039250521 us; error 41 us; skew 500 ppm
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:19.252239 15805 init.cc:377] Logged in from keytab as oryx/127.3.236.188@KRBTEST.COM (short username oryx)
I20260811 07:07:19.252573 15805 webserver.cc:460] Webserver started at http://127.3.236.188:44227/ using document root <none> and password file <none>
I20260811 07:07:19.252700 15805 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.252737 15805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.253541 15805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:19.254108 16027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.254262 15805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.254321 15805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "38490b4a056c406a939544918a5b6aa9"
format_stamp: "Formatted at 2026-08-11 07:07:19 on dist-test-slave-2d19"
server_key: "ee0179ca23e1944d37a92ab5fab3eafa"
server_key_iv: "70ac65dc181c334d0870b817f615f446"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.254386 15805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.269275 15805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:19.271385 15805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:19.275079 15805 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.188:44219
I20260811 07:07:19.275206 16118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.188:44219 every 8 connection(s)
I20260811 07:07:19.275547 15805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
I20260811 07:07:19.276568 16120 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:07:19.277226 16120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Bootstrap starting.
I20260811 07:07:19.279978 16120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:07:19.280368 16120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Bootstrap complete.
I20260811 07:07:19.280680 16120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
I20260811 07:07:19.280913 16120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 38490b4a056c406a939544918a5b6aa9, State: Initialized, Role: LEARNER
I20260811 07:07:19.281008 16120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } }
I20260811 07:07:19.281301 16120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:19.281330 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.281457 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: LEARNER
I20260811 07:07:19.283170 16147 catalog_manager.cc:1259] Loaded cluster ID: 18cc8b0644b14921b74dec4700c9a05f
I20260811 07:07:19.283226 16147 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: loading cluster ID for follower catalog manager: success
I20260811 07:07:19.284989 16147 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: acquiring CA information for follower catalog manager: success
I20260811 07:07:19.285324 16147 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 07:07:19.288492 15976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:19.288686 15976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:19.288710 15976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:19.289829 15976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:19.289896 15976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:19.289954 15976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:19.291517 15976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:33919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=42445
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:19.291783 15976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:19.292047 15976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:19.297124 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.297398 16163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.297144 16165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.297392 15976 server_base.cc:1034] running on GCE node
I20260811 07:07:19.297678 15976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:19.297955 15976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:19.299134 15976 hybrid_clock.cc:648] HybridClock initialized: now 1786432039299112 us; error 40 us; skew 500 ppm
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:19.302237 15976 init.cc:377] Logged in from keytab as oryx/127.3.236.129@KRBTEST.COM (short username oryx)
I20260811 07:07:19.302793 15976 webserver.cc:460] Webserver started at http://127.3.236.129:42445/ using document root <none> and password file <none>
I20260811 07:07:19.303061 15976 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.303158 15976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.304724 15976 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:19.305322 16175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.305528 15976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:19.305606 15976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "15322815fc094adca693bd9b455d756e"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "7ebb7f01a21235052bcb55d45bafa5d3"
server_key_iv: "6ba8925ab0a7b0efe7fef0cb1e636f01"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.305996 15976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.317058 15976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:19.318998 15976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:19.319113 15976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:19.319332 15976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:19.319603 15976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:19.319636 15976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.319659 15976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:19.319672 15976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.326567 15976 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:33919
I20260811 07:07:19.326676 16363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:33919 every 8 connection(s)
I20260811 07:07:19.327039 15976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:19.334817 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 15976
I20260811 07:07:19.335008 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 14994
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:19.349500 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.327806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:44489 (local address 127.3.236.189:35729)
0811 07:07:19.327857 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.327859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.328420 (+ 561us) server_negotiation.cc:407] Connection header received
0811 07:07:19.329598 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.329600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.329615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.329691 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.330634 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331420 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.332632 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.332925 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333007 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.343613 (+ 10606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.343615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.343616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.343628 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.345605 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.347216 (+ 1611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.347218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.347219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.347299 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.347711 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.347712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.347713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.347947 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.348022 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.349223 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.349347 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:07:19.350304 16373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.327810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.3.236.129:44489)
0811 07:07:19.328273 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.328317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.329479 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.329783 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.329789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.330315 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.330511 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.330518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331546 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.331547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.332537 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.332540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.333020 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.333202 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.333893 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.333903 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.343000 (+ 9097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.345749 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.345766 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.345770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.346269 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.347390 (+ 1121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.347391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.347392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.347611 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.348069 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.348079 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.348368 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.349639 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.349928 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":80,"spinlock_wait_cycles":7552,"thread_start_us":363,"threads_started":1}
I20260811 07:07:19.350834 16370 heartbeater.cc:347] Connected to a master server at 127.3.236.189:35729
I20260811 07:07:19.350940 16370 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.351158 16370 heartbeater.cc:511] Master 127.3.236.189:35729 requested a full tablet report, sending...
I20260811 07:07:19.351606 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.328422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.3.236.129:47211)
0811 07:07:19.328585 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.328595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.329558 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.329848 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.329850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.330291 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.330510 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.330515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331467 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.331468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.332654 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.332657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.333428 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.333495 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.334049 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.334054 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.342204 (+ 8150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.348571 (+ 6367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.348572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.348572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.348884 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.350634 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.350635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.350635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.350736 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.351159 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.351160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.351222 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.351286 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.351428 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":82,"thread_start_us":118,"threads_started":1}
I20260811 07:07:19.351969 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.328616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:47211 (local address 127.3.236.188:44219)
0811 07:07:19.328731 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.328737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.328745 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:19.329705 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.329706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.329722 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.329789 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.330625 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331299 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333147 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.333347 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.334027 (+ 680us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.342347 (+ 8320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.342349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.342350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.342359 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.347086 (+ 4727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.349011 (+ 1925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.349012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.349013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.349085 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.350837 (+ 1752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.350838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.350838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.351050 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.351695 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.351745 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.351852 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":61,"threads_started":1}
I20260811 07:07:19.352025 15702 ts_manager.cc:194] Registered new tserver with Master: 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:19.352464 16364 heartbeater.cc:347] Connected to a master server at 127.3.236.188:44219
I20260811 07:07:19.352520 16364 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.352608 16364 heartbeater.cc:511] Master 127.3.236.188:44219 requested a full tablet report, sending...
I20260811 07:07:19.353075 16040 ts_manager.cc:194] Registered new tserver with Master: 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:19.354564 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.327812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:41721 (local address 127.3.236.190:42057)
0811 07:07:19.327914 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.327916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.328492 (+ 576us) server_negotiation.cc:407] Connection header received
0811 07:07:19.329682 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.329684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.329699 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.329730 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.330633 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331420 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.332743 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.333028 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333097 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.351282 (+ 18185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.351285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.351285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.351296 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.353399 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.353790 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.353791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.353791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.353844 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.354019 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.354019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.354020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.354142 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.354186 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.354355 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.354437 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 07:07:19.354964 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.327810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:41721)
0811 07:07:19.328414 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.328427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.329558 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.329811 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.329812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.330314 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.330511 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.330518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.331498 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.331499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.332654 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.332657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.333108 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.333109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.333202 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.333891 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.333903 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.351143 (+ 17240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.353491 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.353493 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.353493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.353703 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.353912 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.353913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.353913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.353962 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.354212 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.354213 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.354249 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.354691 (+ 442us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.354800 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":139,"thread_start_us":367,"threads_started":1}
I20260811 07:07:19.355530 16367 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:19.355602 16367 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.355693 16367 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:19.355970 14658 ts_manager.cc:194] Re-registered known tserver with Master: 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:19.356434 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:41721
I20260811 07:07:19.366405 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:35569
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:42057
--webserver_port=46599
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:19.451820 16383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:19.452091 16383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:19.452128 16383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:19.453322 16383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:19.453403 16383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:19.453451 16383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:19.455155 16383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:35569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=46599
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:19.455430 16383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:19.455672 16383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:19.461666 16399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.461831 16395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.461691 16396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.463006 16383 server_base.cc:1034] running on GCE node
I20260811 07:07:19.463275 16383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:19.463584 16383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:19.464743 16383 hybrid_clock.cc:648] HybridClock initialized: now 1786432039464729 us; error 30 us; skew 500 ppm
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:19.470175 16383 init.cc:377] Logged in from keytab as oryx/127.3.236.130@KRBTEST.COM (short username oryx)
I20260811 07:07:19.470762 16383 webserver.cc:460] Webserver started at http://127.3.236.130:46599/ using document root <none> and password file <none>
I20260811 07:07:19.471030 16383 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.471127 16383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.472800 16383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:19.473338 16414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.473511 16383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:19.473613 16383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "8ac1b62c4f374119a380b0f741347d63"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "cfa5ad85954e31d0c0900797b3c6082c"
server_key_iv: "7b0af9857f14842cf2483deaf2c49c70"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.473976 16383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.501137 16383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:19.503163 16383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:19.503322 16383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:19.503571 16383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:19.503899 16383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:19.503950 16383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.503991 16383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:19.504022 16383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.511062 16383 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:35569
I20260811 07:07:19.511106 16623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:35569 every 8 connection(s)
I20260811 07:07:19.511487 16383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:19.512702 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 16383
I20260811 07:07:19.512884 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 15237
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
I20260811 07:07:19.526943 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:33193 (local address 127.3.236.188:44219)
0811 07:07:19.512424 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.512426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.513818 (+ 1392us) server_negotiation.cc:407] Connection header received
0811 07:07:19.513906 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.513907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.513918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.513945 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.515229 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.515951 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.518070 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.518437 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.518536 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.522964 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.522965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.522966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.522974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.525133 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.525733 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.525734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.525734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.525821 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.526142 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.526142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.526143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.526292 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.526352 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.526746 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.526842 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:19.527941 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.3.236.130:33193)
0811 07:07:19.513768 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.513777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.513841 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.514095 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.514099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.514788 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.515117 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.515122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.516096 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.516097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.517970 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.517974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.518621 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.518623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.518814 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.520649 (+ 1835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.520658 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.522842 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.525208 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.525218 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.525221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.525602 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.525903 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.525904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.525904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.526057 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.526390 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.526399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.526578 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.527404 (+ 826us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.527636 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1445,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260811 07:07:19.528666 16624 heartbeater.cc:347] Connected to a master server at 127.3.236.188:44219
I20260811 07:07:19.528781 16624 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.528832 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:46375
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:42057
--webserver_port=37143
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:07:19.529003 16624 heartbeater.cc:511] Master 127.3.236.188:44219 requested a full tablet report, sending...
I20260811 07:07:19.529526 16040 ts_manager.cc:194] Registered new tserver with Master: 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130:35569)
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:19.530665 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:43243 (local address 127.3.236.190:42057)
0811 07:07:19.512752 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.512754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.521450 (+ 8696us) server_negotiation.cc:407] Connection header received
0811 07:07:19.521558 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.521559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.521572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.521605 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.522168 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.522883 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.524050 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.524347 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.524422 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.527530 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.527532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.527532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.527540 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.529505 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.529900 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.529901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.529901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.529973 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.530159 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.530160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.530160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.530277 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.530362 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.530496 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.530575 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 07:07:19.530686 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:43243)
0811 07:07:19.521367 (+ 8758us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.521380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.521482 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.521694 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.521694 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.521983 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.522062 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.522065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.523015 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.523016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.523923 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.523926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.525008 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.525010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.525128 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.525740 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.525745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.527417 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.529596 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.529597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.529597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.529813 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.530045 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.530045 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.530045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.530091 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.530352 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.530353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.530395 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.530466 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.530568 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8714,"mutex_wait_us":29,"thread_start_us":60,"threads_started":1}
I20260811 07:07:19.531334 16625 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:19.531388 16625 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.531659 16625 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:19.531955 14658 ts_manager.cc:194] Re-registered known tserver with Master: 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130:35569)
I20260811 07:07:19.532421 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:43243
I20260811 07:07:19.533703 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:34915 (local address 127.3.236.189:35729)
0811 07:07:19.513428 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.513430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.513443 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:07:19.523074 (+ 9631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.523076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.523089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.523145 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.524130 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.524881 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.526562 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.526889 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.526930 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.530632 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.530634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.530634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.530644 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.532409 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.532835 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.532836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.532836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.532900 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.533123 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.533124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.533124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.533288 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.533357 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.533524 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.533605 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1115}
I20260811 07:07:19.533944 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.512287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.3.236.130:34915)
0811 07:07:19.512799 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.512852 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.513601 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.523196 (+ 9595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.523197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.523677 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.523741 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.523743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.524988 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.524990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.526445 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.526448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.527012 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.527013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.527087 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.528223 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.528228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.530515 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.532541 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.532542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.532543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.532753 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.532986 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.532987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.532987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.533051 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.533373 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.533374 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.533411 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.533490 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.533601 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":513,"spinlock_wait_cycles":8576,"thread_start_us":94,"threads_started":1}
I20260811 07:07:19.534363 16626 heartbeater.cc:347] Connected to a master server at 127.3.236.189:35729
I20260811 07:07:19.534431 16626 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.534502 16626 heartbeater.cc:511] Master 127.3.236.189:35729 requested a full tablet report, sending...
I20260811 07:07:19.534859 15702 ts_manager.cc:194] Registered new tserver with Master: 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130:35569)
I20260811 07:07:19.581220 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.578476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49816 (local address 127.3.236.188:44219)
0811 07:07:19.578524 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.578526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.578541 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:07:19.578569 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.578570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.578601 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.578675 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.578848 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.579454 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.580446 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.580871 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.580957 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.581034 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.581095 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.581136 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:19.581220 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.578237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.0.0.1:49816)
0811 07:07:19.578295 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.578307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.578378 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.578698 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.578699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.578702 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.578758 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.578760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.579545 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.579546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.580340 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.580343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.580960 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.580962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.581024 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.581028 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.581082 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.581140 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:07:19.581537 16083 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 07:07:19.581710 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e196cba1714bc5ad86f0de1abc2994. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.581830 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: LEARNER
W20260811 07:07:19.617689 16632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:19.617878 16632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:19.617900 16632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:19.619061 16632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:19.619103 16632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:19.619153 16632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:19.620769 16632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:46375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=37143
--tserver_master_addrs=127.3.236.190:42057,127.3.236.189:35729,127.3.236.188:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:19.621028 16632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:19.621274 16632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:19.626608 16644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:19.626665 16647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.626632 16632 server_base.cc:1034] running on GCE node
W20260811 07:07:19.626705 16645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:19.627301 16632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:19.627552 16632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:19.628719 16632 hybrid_clock.cc:648] HybridClock initialized: now 1786432039628714 us; error 36 us; skew 500 ppm
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:19.631214 16632 init.cc:377] Logged in from keytab as oryx/127.3.236.131@KRBTEST.COM (short username oryx)
I20260811 07:07:19.631650 16632 webserver.cc:460] Webserver started at http://127.3.236.131:37143/ using document root <none> and password file <none>
I20260811 07:07:19.631914 16632 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:19.632005 16632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:19.633407 16632 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:19.634105 16658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.634291 16632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:19.634400 16632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "41758d2ff486493987ab4e49efa9b070"
format_stamp: "Formatted at 2026-08-11 07:07:18 on dist-test-slave-2d19"
server_key: "68bcbf2e01187610746ae3af28f0b666"
server_key_iv: "d90b7967b84ec71f30d3613349a2e92d"
server_key_version: "encryptionkey@0"
I20260811 07:07:19.634780 16632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:19.698446 16632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:19.700860 16632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:19.701021 16632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:19.701320 16632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:19.701656 16632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:19.701692 16632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.701715 16632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:19.701762 16632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:19.707827 16632 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:46375
I20260811 07:07:19.707872 16858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:46375 every 8 connection(s)
I20260811 07:07:19.708352 16632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.188@KRBTEST.COM
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:19.716239 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 16632
Aug 11 07:07:19 dist-test-slave-2d19 krb5kdc[14570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.189@KRBTEST.COM
I20260811 07:07:19.720983 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:43749 (local address 127.3.236.189:35729)
0811 07:07:19.709237 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.709240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.709504 (+ 264us) server_negotiation.cc:407] Connection header received
0811 07:07:19.710799 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.710800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.710811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.710868 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.711721 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712531 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713663 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.713868 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713899 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.717292 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.717294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.717294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.717302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.719215 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.719857 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.719859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.719859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.719918 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.720213 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.720214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.720214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.720364 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.720406 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.720801 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.720884 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:19.721776 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.3.236.131:43749)
0811 07:07:19.709335 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.709379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.710681 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.710950 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.710956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.711401 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.711605 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.711611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712631 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.712632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.713554 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.713556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.713994 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.714056 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.714481 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.714486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.717191 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.719411 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.719412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.719412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.719769 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.720003 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.720003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.720004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.720137 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.720451 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.720458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.720643 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.721246 (+ 603us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.721433 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":44,"thread_start_us":121,"threads_started":1}
I20260811 07:07:19.722015 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:48063 (local address 127.3.236.190:42057)
0811 07:07:19.709185 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.709187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.709714 (+ 527us) server_negotiation.cc:407] Connection header received
0811 07:07:19.710949 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.710950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.710963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.711056 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.711722 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712532 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713672 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.713963 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.714069 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.716836 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.716838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.716838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.716846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.719387 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.720827 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.720828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.720828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.720889 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.721113 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.721114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.721115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.721229 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.721282 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.721835 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.721925 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:07:19.721776 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:48063)
0811 07:07:19.709649 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.709660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.710693 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.711066 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.711066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.711415 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.711605 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.711611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712631 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.712632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.713559 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.713560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.714072 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.714073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.714147 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.714649 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.714654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.716725 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.719496 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.719498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.719498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.720749 (+ 1251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.720970 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.720971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.720972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.721040 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.721305 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.721306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.721345 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.721402 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.721514 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":81,"thread_start_us":105,"threads_started":1}
I20260811 07:07:19.722633 16863 heartbeater.cc:347] Connected to a master server at 127.3.236.189:35729
I20260811 07:07:19.722683 16861 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42057
I20260811 07:07:19.722739 16863 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.722739 16861 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.722977 16861 heartbeater.cc:511] Master 127.3.236.190:42057 requested a full tablet report, sending...
I20260811 07:07:19.723349 14658 ts_manager.cc:194] Re-registered known tserver with Master: 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375)
I20260811 07:07:19.723815 14658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:48063
I20260811 07:07:19.724592 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:56675 (local address 127.3.236.188:44219)
0811 07:07:19.709741 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.709743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.710966 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 07:07:19.710997 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.710998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.711009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.711057 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.711704 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712218 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713325 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.713527 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713599 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.716174 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.716176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.716176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.716184 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.718138 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.723673 (+ 5535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.723675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.723675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.723738 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.723996 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.723997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.723998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.724141 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.724189 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.724423 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.724510 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:07:19.722986 16863 heartbeater.cc:511] Master 127.3.236.189:35729 requested a full tablet report, sending...
I20260811 07:07:19.724942 15702 ts_manager.cc:194] Registered new tserver with Master: 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375)
I20260811 07:07:19.725144 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.709503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.3.236.131:56675)
0811 07:07:19.710868 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.710878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.710927 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.711109 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.711111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.711372 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:19.711605 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.711613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.712391 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.712392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.713228 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.713230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.713621 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.713622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.713755 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.714358 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:19.714367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:19.716017 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:19.718214 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.718222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.718225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.722702 (+ 4477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.723810 (+ 1108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:19.723811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:19.723811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:19.723888 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:19.724245 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:19.724246 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:19.724303 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.724711 (+ 408us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.724840 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260811 07:07:19.725432 16859 heartbeater.cc:347] Connected to a master server at 127.3.236.188:44219
I20260811 07:07:19.725485 16859 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:19.725572 16859 heartbeater.cc:511] Master 127.3.236.188:44219 requested a full tablet report, sending...
I20260811 07:07:19.725859 16040 ts_manager.cc:194] Registered new tserver with Master: 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375)
I20260811 07:07:19.727421 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.718078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.3.236.189:35729)
0811 07:07:19.718734 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.718736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.718906 (+ 170us) server_negotiation.cc:407] Connection header received
0811 07:07:19.718928 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.718929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.718940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.719005 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.719731 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.720502 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.721672 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.722093 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.722169 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.723884 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.723885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.723886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.723893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.726257 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.726667 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.726668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.726669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.726725 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.726910 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.726911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.726912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.727026 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.727131 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.727236 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.727314 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":63,"threads_started":1}
I20260811 07:07:19.729337 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.717661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.3.236.190:42057)
0811 07:07:19.718682 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.718684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.718800 (+ 116us) server_negotiation.cc:407] Connection header received
0811 07:07:19.718827 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.718828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.718842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.718901 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.719657 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.720512 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.723642 (+ 3130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.724026 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.724093 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.726219 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.726220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.726221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.726228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.728103 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.728497 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.728498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.728499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.728562 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.728789 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.728790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.728790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.728933 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.729049 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.729131 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.729220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":71,"threads_started":1}
I20260811 07:07:19.730723 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.719316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.3.236.188:44219)
0811 07:07:19.719898 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.719900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.719907 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:07:19.719929 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.719929 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.719940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.719993 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.720694 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.721441 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.722461 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.725465 (+ 3004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.725531 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.726879 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.726880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.726881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.726888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.729424 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.729825 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.729826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.729826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.729913 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.730124 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.730124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.730124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.730282 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.730390 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.730549 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.730632 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":55,"threads_started":1}
I20260811 07:07:19.731263 14658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:19.733366 14658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:19.734228 15986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: NON_VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: true } } }
I20260811 07:07:19.734357 15986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.735543 16897 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: attempting to promote NON_VOTER 38490b4a056c406a939544918a5b6aa9 to VOTER
I20260811 07:07:19.735714 16897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } }
I20260811 07:07:19.736397 16083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 LEARNER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 07:07:19.736658 15722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Refusing update from remote peer 81e196cba1714bc5ad86f0de1abc2994: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 07:07:19.736757 15986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 07:07:19.736876 15986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 07:07:19.737540 16895 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 38490b4a056c406a939544918a5b6aa9 (127.3.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.737782 16083 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 38490b4a056c406a939544918a5b6aa9 (127.3.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.738622 16891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.738730 16891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.738842 16891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.738900 16891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.738929 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.739015 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:07:19.739102 15722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 38490b4a056c406a939544918a5b6aa9 (127.3.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.738638 16895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.739216 16895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:19.741005 16912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81e196cba1714bc5ad86f0de1abc2994" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:19.741285 16912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:07:19.744800 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.740562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.3.236.130:35569)
0811 07:07:19.740743 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.740746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.740756 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:19.740800 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.740802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.740819 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.740927 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.741173 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742140 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.743561 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744372 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.744483 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.744576 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.744648 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.744695 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260811 07:07:19.744813 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.739931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:46375 (local address 127.0.0.1:49210)
0811 07:07:19.739966 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.739980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.740024 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.740939 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.740941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.740943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.740997 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.740999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742330 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.742331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.743456 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.743458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744554 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.744555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.744634 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.744637 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.744670 (+ 33us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.744710 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":30}
I20260811 07:07:19.744833 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.740006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33919 (local address 127.0.0.1:41168)
0811 07:07:19.740143 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.740153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.740222 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.741596 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.741597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.741599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.741640 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.741641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742884 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.742885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.743780 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.743783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744573 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.744574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.744652 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.744656 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.744684 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.744715 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260811 07:07:19.745031 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.740470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41168 (local address 127.3.236.129:33919)
0811 07:07:19.741427 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.741432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.741443 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:19.741481 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.741483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.741499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.741580 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.741711 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742790 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.743866 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744478 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.744759 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.744850 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.744877 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.744926 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":93,"threads_started":1}
I20260811 07:07:19.745043 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.740190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:35569 (local address 127.0.0.1:57426)
0811 07:07:19.740537 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.740548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.740610 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.741079 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.741080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.741082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.741114 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.741115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742226 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.742227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.743456 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.743458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744493 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.744494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.744564 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.744570 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.744632 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.744680 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":26,"thread_start_us":100,"threads_started":1}
I20260811 07:07:19.745251 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.740107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.3.236.131:46375)
0811 07:07:19.740738 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.740745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.740756 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:19.740800 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.740802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.740820 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.740908 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.741074 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.742199 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.743560 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.744461 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.745012 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.745108 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.745134 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.745176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":190,"threads_started":1}
I20260811 07:07:19.745857 16750 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae28872f199499fb8e95586bb9cee1a (DEFAULT_TABLE table=test-table [id=a4608a23c0ec4e0d9e9b684c0e5407af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:19.745864 16260 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae28872f199499fb8e95586bb9cee1a (DEFAULT_TABLE table=test-table [id=a4608a23c0ec4e0d9e9b684c0e5407af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:19.746269 16260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae28872f199499fb8e95586bb9cee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:19.746259 16750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae28872f199499fb8e95586bb9cee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:19.748970 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 7ae28872f199499fb8e95586bb9cee1a (DEFAULT_TABLE table=test-table [id=a4608a23c0ec4e0d9e9b684c0e5407af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:19.749302 16926 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Bootstrap starting.
I20260811 07:07:19.749346 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ae28872f199499fb8e95586bb9cee1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:19.749559 16927 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Bootstrap starting.
I20260811 07:07:19.750140 16926 tablet_bootstrap.cc:654] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:19.750234 16927 tablet_bootstrap.cc:654] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:19.750559 16926 log.cc:824] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:19.750581 16927 log.cc:824] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:19.751387 16927 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: No bootstrap required, opened a new log
I20260811 07:07:19.751461 16927 ts_tablet_manager.cc:1397] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:07:19.751679 16926 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: No bootstrap required, opened a new log
I20260811 07:07:19.751737 16926 ts_tablet_manager.cc:1397] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:19.752794 16930 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Bootstrap starting.
I20260811 07:07:19.753424 16930 tablet_bootstrap.cc:654] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:19.753782 16930 log.cc:824] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:19.754374 16926 raft_consensus.cc:348] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.754549 16926 raft_consensus.cc:374] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:19.754587 16926 raft_consensus.cc:729] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15322815fc094adca693bd9b455d756e, State: Initialized, Role: FOLLOWER
I20260811 07:07:19.754541 16927 raft_consensus.cc:348] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.754699 16927 raft_consensus.cc:374] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:19.754739 16927 raft_consensus.cc:729] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41758d2ff486493987ab4e49efa9b070, State: Initialized, Role: FOLLOWER
I20260811 07:07:19.754767 16930 tablet_bootstrap.cc:492] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: No bootstrap required, opened a new log
I20260811 07:07:19.754735 16926 consensus_queue.cc:260] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.754868 16930 ts_tablet_manager.cc:1397] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:07:19.754849 16927 consensus_queue.cc:260] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.755023 16926 ts_tablet_manager.cc:1428] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:07:19.755188 16367 heartbeater.cc:503] Master 127.3.236.190:42057 was elected leader, sending a full tablet report...
I20260811 07:07:19.755102 16927 ts_tablet_manager.cc:1428] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 07:07:19.756702 16861 heartbeater.cc:503] Master 127.3.236.190:42057 was elected leader, sending a full tablet report...
I20260811 07:07:19.757869 16930 raft_consensus.cc:348] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.758020 16930 raft_consensus.cc:374] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:19.758068 16930 raft_consensus.cc:729] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac1b62c4f374119a380b0f741347d63, State: Initialized, Role: FOLLOWER
I20260811 07:07:19.758180 16930 consensus_queue.cc:260] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.758445 16930 ts_tablet_manager.cc:1428] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 07:07:19.758512 16625 heartbeater.cc:503] Master 127.3.236.190:42057 was elected leader, sending a full tablet report...
W20260811 07:07:19.762234 16628 tablet.cc:2369] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:07:19.777741 16935 raft_consensus.cc:482] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:19.777887 16935 raft_consensus.cc:504] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.778285 16935 leader_election.cc:288] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375), 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:19.781982 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.778683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:35891 (local address 127.3.236.131:46375)
0811 07:07:19.778754 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.778756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.778768 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:19.778789 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.778790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.778797 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.778876 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.779103 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.779798 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781021 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.781633 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781698 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.781705 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.781845 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.781889 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:07:19.781961 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.778519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:53943 (local address 127.3.236.129:33919)
0811 07:07:19.778565 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.778567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.778583 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:07:19.778607 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.778608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.778615 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.778663 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:19.778842 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.779621 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781021 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.781588 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781669 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.781679 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.781846 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.781889 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:07:19.781992 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.778472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:46375 (local address 127.3.236.130:35891)
0811 07:07:19.778521 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.778530 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.778565 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.778898 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.778899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.778900 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.779020 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.779021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.779866 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.779867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.780942 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.780943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.781691 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.781772 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.781775 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.781827 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.781857 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:07:19.781970 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.778436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33919 (local address 127.3.236.130:53943)
0811 07:07:19.778494 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:19.778505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:19.778549 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:19.778685 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:19.778686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:19.778688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:19.778769 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.778770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.779707 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.779708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.780906 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:19.780908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.781666 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.781667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:19.781744 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.781748 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:19.781812 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:07:19.781857 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:07:19.782374 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ae28872f199499fb8e95586bb9cee1a" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41758d2ff486493987ab4e49efa9b070" is_pre_election: true
I20260811 07:07:19.782559 16776 raft_consensus.cc:2454] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 0.
I20260811 07:07:19.782680 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ae28872f199499fb8e95586bb9cee1a" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15322815fc094adca693bd9b455d756e" is_pre_election: true
I20260811 07:07:19.782820 16421 leader_election.cc:302] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41758d2ff486493987ab4e49efa9b070, 8ac1b62c4f374119a380b0f741347d63; no voters:
I20260811 07:07:19.782855 16285 raft_consensus.cc:2454] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 0.
I20260811 07:07:19.783069 16935 raft_consensus.cc:2781] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:19.783147 16935 raft_consensus.cc:482] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:19.783177 16935 raft_consensus.cc:3037] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:19.783993 16935 raft_consensus.cc:504] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.784155 16935 leader_election.cc:288] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 election: Requested vote from peers 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375), 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:19.784313 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ae28872f199499fb8e95586bb9cee1a" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15322815fc094adca693bd9b455d756e"
I20260811 07:07:19.784394 16285 raft_consensus.cc:3037] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:19.784369 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ae28872f199499fb8e95586bb9cee1a" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41758d2ff486493987ab4e49efa9b070"
I20260811 07:07:19.784482 16776 raft_consensus.cc:3037] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:19.785362 16285 raft_consensus.cc:2454] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 1.
I20260811 07:07:19.785493 16776 raft_consensus.cc:2454] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 1.
I20260811 07:07:19.785542 16420 leader_election.cc:302] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15322815fc094adca693bd9b455d756e, 8ac1b62c4f374119a380b0f741347d63; no voters:
I20260811 07:07:19.785630 16935 raft_consensus.cc:2781] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:19.785760 16935 raft_consensus.cc:686] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac1b62c4f374119a380b0f741347d63, State: Running, Role: LEADER
I20260811 07:07:19.785872 16935 consensus_queue.cc:237] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } }
I20260811 07:07:19.786950 14658 catalog_manager.cc:5495] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "8ac1b62c4f374119a380b0f741347d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } health_report { overall_health: HEALTHY } } }
I20260811 07:07:19.801803 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.799003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.3.236.130:35569)
0811 07:07:19.799074 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.799076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.799090 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:07:19.799222 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.799227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.799249 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.799339 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:19.799503 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.800108 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.800960 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.801163 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.801256 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.801424 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:19.801536 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:19.801606 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.801646 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.801668 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:07:19.803722 16285 raft_consensus.cc:1261] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Refusing update from remote peer 8ac1b62c4f374119a380b0f741347d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:19.803723 16776 raft_consensus.cc:1261] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Refusing update from remote peer 8ac1b62c4f374119a380b0f741347d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:19.804085 16935 consensus_queue.cc:1035] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:19.804251 16937 consensus_queue.cc:1035] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:19.807515 16942 mvcc.cc:205] Tried to move back new op lower bound from 7317225635033174016 to 7317225634963435520. Current Snapshot: MvccSnapshot[applied={T|T < 7317225635033174016}]
I20260811 07:07:19.808947 16943 mvcc.cc:205] Tried to move back new op lower bound from 7317225635033174016 to 7317225634963435520. Current Snapshot: MvccSnapshot[applied={T|T < 7317225635033174016}]
I20260811 07:07:19.811371 14658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:19.811480 14658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:19.812847 14658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81e196cba1714bc5ad86f0de1abc2994: Sending DeleteTablet for 3 replicas of tablet 7ae28872f199499fb8e95586bb9cee1a
I20260811 07:07:19.813246 16260 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ae28872f199499fb8e95586bb9cee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:19 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:41168
I20260811 07:07:19.813292 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ae28872f199499fb8e95586bb9cee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:19 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:57426
I20260811 07:07:19.813535 16961 tablet_replica.cc:330] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: stopping tablet replica
I20260811 07:07:19.813643 16961 raft_consensus.cc:2229] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:19.813750 16750 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ae28872f199499fb8e95586bb9cee1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:19 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:49210
I20260811 07:07:19.814020 16961 raft_consensus.cc:2258] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:19.814491 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 14597
I20260811 07:07:19.821782 16961 ts_tablet_manager.cc:1905] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:19.822937 16965 tablet_replica.cc:330] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: stopping tablet replica
I20260811 07:07:19.823052 16965 raft_consensus.cc:2229] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:19.823181 16965 raft_consensus.cc:2258] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:19.823876 16961 ts_tablet_manager.cc:1918] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:19.823943 16961 log.cc:1196] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/7ae28872f199499fb8e95586bb9cee1a
I20260811 07:07:19.824218 16961 ts_tablet_manager.cc:1939] T 7ae28872f199499fb8e95586bb9cee1a P 8ac1b62c4f374119a380b0f741347d63: Deleting consensus metadata
I20260811 07:07:19.824738 16960 tablet_replica.cc:330] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: stopping tablet replica
I20260811 07:07:19.824851 16960 raft_consensus.cc:2229] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:19.824957 16960 raft_consensus.cc:2258] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:19.825067 16965 ts_tablet_manager.cc:1905] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:19.825407 16960 ts_tablet_manager.cc:1905] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:19.826841 16965 ts_tablet_manager.cc:1918] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:19.826908 16965 log.cc:1196] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/7ae28872f199499fb8e95586bb9cee1a
I20260811 07:07:19.827159 16960 ts_tablet_manager.cc:1918] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:19.827203 16965 ts_tablet_manager.cc:1939] T 7ae28872f199499fb8e95586bb9cee1a P 41758d2ff486493987ab4e49efa9b070: Deleting consensus metadata
I20260811 07:07:19.827224 16960 log.cc:1196] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/7ae28872f199499fb8e95586bb9cee1a
I20260811 07:07:19.827503 16960 ts_tablet_manager.cc:1939] T 7ae28872f199499fb8e95586bb9cee1a P 15322815fc094adca693bd9b455d756e: Deleting consensus metadata
W20260811 07:07:19.829399 16181 connection.cc:535] server connection from 127.0.0.1:41168 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 07:07:19.829454 16418 connection.cc:535] server connection from 127.0.0.1:57426 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 07:07:19.829654 16666 connection.cc:535] server connection from 127.0.0.1:49210 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:07:19.833920 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.813786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.3.236.188:44219)
0811 07:07:19.813825 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.813827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.814456 (+ 629us) server_negotiation.cc:407] Connection header received
0811 07:07:19.814562 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.814563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.814575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.814604 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.823847 (+ 9243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.824643 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.828499 (+ 3856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.828761 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.828850 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.830400 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.830402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.830402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.830410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.832291 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.832783 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.832784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.832784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.832847 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.833059 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.833059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.833060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.833193 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.833272 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.833719 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.833819 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:07:19.840034 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.831667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.3.236.189:35729)
0811 07:07:19.831722 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.831724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.831980 (+ 256us) server_negotiation.cc:407] Connection header received
0811 07:07:19.832089 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.832111 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.832126 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.832162 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.833393 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.834083 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.835017 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.835254 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.835335 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.836602 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.836603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.836604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.836611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.838489 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.838891 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.838893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.838893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.838956 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.839257 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.839258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.839259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.839404 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.839577 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.839722 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.839908 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 07:07:19.840330 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:19.831930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.3.236.188:44219)
0811 07:07:19.831979 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:19.831981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:19.832132 (+ 151us) server_negotiation.cc:407] Connection header received
0811 07:07:19.832225 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:19.832226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:19.832236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:19.832293 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:19.833107 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.833876 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.834935 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:19.835252 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:19.835337 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:19.837202 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:19.837203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:19.837204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:19.837211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:19.838972 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.839366 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.839367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.839367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.839419 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:19.839650 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:19.839651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:19.839652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:19.839775 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:19.839876 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:19.840088 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 07:07:19.840211 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:07:20.809124 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:20.808786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:39677)
0811 07:07:20.808968 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:20.809064 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260811 07:07:20.809523 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:20.809290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:44783)
0811 07:07:20.809391 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:20.809470 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
W20260811 07:07:20.809549 16861 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:42057 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
W20260811 07:07:20.809764 16367 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:42057 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:20.809804 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:20.809028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:35435)
0811 07:07:20.809689 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:20.809761 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":62,"threads_started":1}
W20260811 07:07:20.809944 16625 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:42057 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.313570 16980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 81e196cba1714bc5ad86f0de1abc2994)
I20260811 07:07:21.313771 16980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } }
I20260811 07:07:21.314095 16980 leader_election.cc:288] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057), 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189:35729)
I20260811 07:07:21.314492 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.314256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40900)
0811 07:07:21.314376 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.314436 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":51,"threads_started":1}
W20260811 07:07:21.314703 16029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.315145 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.315038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40916)
0811 07:07:21.315084 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.315117 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 07:07:21.315126 16987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 81e196cba1714bc5ad86f0de1abc2994)
W20260811 07:07:21.315234 16029 leader_election.cc:334] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057): Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.315356 16987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } }
I20260811 07:07:21.315598 16987 leader_election.cc:288] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057), 38490b4a056c406a939544918a5b6aa9 (127.3.236.188:44219)
I20260811 07:07:21.316013 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.315717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40920)
0811 07:07:21.315834 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.315930 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
W20260811 07:07:21.316217 15676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.316704 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.316593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40922)
0811 07:07:21.316650 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.316678 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 07:07:21.316855 15676 leader_election.cc:334] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057): Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.318013 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.314375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.3.236.189:35729)
0811 07:07:21.314480 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.314482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.314493 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:21.314520 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.314521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.314530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.314632 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:21.314839 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.315608 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.317260 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.317652 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.317735 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.317747 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.317899 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.317941 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260811 07:07:21.318004 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.314269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:35729 (local address 127.0.0.1:37812)
0811 07:07:21.314377 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.314389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:21.314449 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:21.314667 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:21.314669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:21.314671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:21.314732 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.314734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.315761 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.315762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.317188 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.317191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.317736 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.317737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.317802 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.317806 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:21.317891 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 07:07:21.317935 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":46,"threads_started":1}
I20260811 07:07:21.318305 15722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38490b4a056c406a939544918a5b6aa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" is_pre_election: true
I20260811 07:07:21.318388 15722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38490b4a056c406a939544918a5b6aa9 in term 1.
I20260811 07:07:21.318574 16031 leader_election.cc:302] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38490b4a056c406a939544918a5b6aa9, 74a83fc5015d4b17b0e7dcea87b3805f; no voters: 81e196cba1714bc5ad86f0de1abc2994
I20260811 07:07:21.318681 16980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 07:07:21.318746 16980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 81e196cba1714bc5ad86f0de1abc2994)
I20260811 07:07:21.318779 16980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:07:21.319511 16980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } }
I20260811 07:07:21.319666 16980 leader_election.cc:288] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 election: Requested vote from peers 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057), 74a83fc5015d4b17b0e7dcea87b3805f (127.3.236.189:35729)
I20260811 07:07:21.319828 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.319736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40932)
0811 07:07:21.319778 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.319804 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 07:07:21.319850 15722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38490b4a056c406a939544918a5b6aa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "74a83fc5015d4b17b0e7dcea87b3805f"
I20260811 07:07:21.319933 15722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:07:21.320077 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.315887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:44219 (local address 127.0.0.1:49830)
0811 07:07:21.315974 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.315983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:21.316027 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:21.316322 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:21.316323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:21.316325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:21.316368 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.316370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.317186 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.317187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.317910 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.317912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.319830 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.319832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.319903 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.319909 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:07:21.319943 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 07:07:21.319989 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":45,"threads_started":1}
I20260811 07:07:21.320128 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.320029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40936)
0811 07:07:21.320063 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.320104 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260811 07:07:21.320231 16029 leader_election.cc:334] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057): Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
I20260811 07:07:21.320366 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.316026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.3.236.188:44219)
0811 07:07:21.316166 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.316168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.316176 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:21.316196 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.316197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.316207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.316292 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:21.316452 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.317119 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.318030 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.318653 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.320185 (+ 1532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.320201 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.320223 (+ 22us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.320296 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260811 07:07:21.320554 16083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "38490b4a056c406a939544918a5b6aa9" is_pre_election: true
I20260811 07:07:21.320634 16083 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74a83fc5015d4b17b0e7dcea87b3805f in current term 2: Already voted for candidate 38490b4a056c406a939544918a5b6aa9 in this term.
I20260811 07:07:21.320828 15676 leader_election.cc:302] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74a83fc5015d4b17b0e7dcea87b3805f; no voters: 38490b4a056c406a939544918a5b6aa9, 81e196cba1714bc5ad86f0de1abc2994
I20260811 07:07:21.320853 15722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38490b4a056c406a939544918a5b6aa9 in term 2.
I20260811 07:07:21.320928 16987 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:07:21.321038 16031 leader_election.cc:302] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38490b4a056c406a939544918a5b6aa9, 74a83fc5015d4b17b0e7dcea87b3805f; no voters: 81e196cba1714bc5ad86f0de1abc2994
I20260811 07:07:21.321157 16980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:07:21.321318 16980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [term 2 LEADER]: Becoming Leader. State: Replica: 38490b4a056c406a939544918a5b6aa9, State: Running, Role: LEADER
I20260811 07:07:21.321429 16980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } }
I20260811 07:07:21.321810 16994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38490b4a056c406a939544918a5b6aa9. Latest consensus state: current_term: 2 leader_uuid: "38490b4a056c406a939544918a5b6aa9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:21.321892 16994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:21.322062 16996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:21.322522 16996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:21.322640 16996 catalog_manager.cc:1259] Loaded cluster ID: 18cc8b0644b14921b74dec4700c9a05f
I20260811 07:07:21.322674 16996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:21.322835 16996 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:21.322937 16996 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Loaded TSK: 0
I20260811 07:07:21.323078 16996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:21.327261 16040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:21.328987 16040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:21.329895 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.329821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40962)
0811 07:07:21.329847 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.329874 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260811 07:07:21.330135 15722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [term 2 FOLLOWER]: Refusing update from remote peer 38490b4a056c406a939544918a5b6aa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 07:07:21.330329 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.330235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.0.0.1:40974)
0811 07:07:21.330269 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.330299 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 07:07:21.330386 16994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 07:07:21.330505 16029 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 -> Peer 81e196cba1714bc5ad86f0de1abc2994 (127.3.236.190:42057): Couldn't send request to peer 81e196cba1714bc5ad86f0de1abc2994. Status: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 07:07:21.331009 16987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38490b4a056c406a939544918a5b6aa9. Latest consensus state: current_term: 2 leader_uuid: "38490b4a056c406a939544918a5b6aa9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:21.331125 16987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:07:21.331501 16980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "38490b4a056c406a939544918a5b6aa9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:21.331585 16980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:21.331719 16987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "38490b4a056c406a939544918a5b6aa9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "81e196cba1714bc5ad86f0de1abc2994" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42057 } } peers { permanent_uuid: "74a83fc5015d4b17b0e7dcea87b3805f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 35729 } attrs { promote: false } } peers { permanent_uuid: "38490b4a056c406a939544918a5b6aa9" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 44219 } attrs { promote: false } } }
I20260811 07:07:21.331847 16987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a83fc5015d4b17b0e7dcea87b3805f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:07:21.336485 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.3.236.131:46375)
0811 07:07:21.333921 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.333923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.333930 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:07:21.333956 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.333957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.333967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.334029 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:21.334141 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.334629 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.335564 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.336117 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.336199 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.336211 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.336380 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.336417 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":98,"threads_started":1}
I20260811 07:07:21.336479 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:46375 (local address 127.0.0.1:49222)
0811 07:07:21.333438 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.333448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:21.333491 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:21.334029 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:21.334030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:21.334033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:21.334086 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.334088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.334713 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.334714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.335469 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.335471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.336210 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.336211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.336307 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.336311 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:21.336365 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:07:21.336410 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":41}
I20260811 07:07:21.336894 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:35569 (local address 127.0.0.1:57456)
0811 07:07:21.333437 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.333448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:21.333490 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:21.334011 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:21.334012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:21.334014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:21.334046 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.334047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.334707 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.334708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.335861 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.335864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.336669 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.336670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.336729 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.336732 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:21.336786 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:07:21.336830 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":29}
I20260811 07:07:21.336917 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.3.236.130:35569)
0811 07:07:21.333688 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.333690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.333698 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:21.333726 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.333727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.333761 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.334107 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:21.334153 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.334616 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.335964 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.336567 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.336666 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.336679 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.336796 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.336843 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":64,"threads_started":1}
I20260811 07:07:21.336930 16750 tablet_service.cc:1467] Processing CreateTablet for tablet 95235a0e04d2478084310cb45763e12f (DEFAULT_TABLE table=test-table [id=b425147bf0ad466abcdac14446556e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:21.337080 16750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95235a0e04d2478084310cb45763e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:21.337276 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 95235a0e04d2478084310cb45763e12f (DEFAULT_TABLE table=test-table [id=b425147bf0ad466abcdac14446556e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:21.337409 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95235a0e04d2478084310cb45763e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:21.337750 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33919 (local address 127.0.0.1:41174)
0811 07:07:21.333533 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.333544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:21.333596 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:21.333848 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:21.333849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:21.333852 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:21.333904 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.333905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.335586 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.335587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.336664 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:21.336667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.337505 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.337506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:21.337573 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.337577 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:21.337613 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 07:07:21.337690 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260811 07:07:21.338114 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.333585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.3.236.129:33919)
0811 07:07:21.333722 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.333724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.333733 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:21.333761 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.333762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.333771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.333837 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:21.334727 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.335485 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.336760 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.337396 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.337906 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.337920 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.337982 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.338021 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260811 07:07:21.338433 16260 tablet_service.cc:1467] Processing CreateTablet for tablet 95235a0e04d2478084310cb45763e12f (DEFAULT_TABLE table=test-table [id=b425147bf0ad466abcdac14446556e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:21.338560 16260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95235a0e04d2478084310cb45763e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:21.338694 17020 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Bootstrap starting.
I20260811 07:07:21.339171 17020 tablet_bootstrap.cc:654] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:21.339490 17022 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Bootstrap starting.
I20260811 07:07:21.339769 17020 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: No bootstrap required, opened a new log
I20260811 07:07:21.339812 17020 ts_tablet_manager.cc:1397] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:21.339991 17020 raft_consensus.cc:348] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.340049 17030 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: Bootstrap starting.
I20260811 07:07:21.340095 17020 raft_consensus.cc:374] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:21.340122 17020 raft_consensus.cc:729] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41758d2ff486493987ab4e49efa9b070, State: Initialized, Role: FOLLOWER
I20260811 07:07:21.340183 17020 consensus_queue.cc:260] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.340384 17020 ts_tablet_manager.cc:1428] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:21.340516 17030 tablet_bootstrap.cc:654] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:21.340665 17022 tablet_bootstrap.cc:654] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:21.340929 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.340689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:47439)
0811 07:07:21.340834 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.340878 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260811 07:07:21.341205 16859 heartbeater.cc:503] Master 127.3.236.188:44219 was elected leader, sending a full tablet report...
I20260811 07:07:21.341509 17030 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: No bootstrap required, opened a new log
I20260811 07:07:21.341594 17030 ts_tablet_manager.cc:1397] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:21.341722 17030 raft_consensus.cc:348] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.341771 17022 tablet_bootstrap.cc:492] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: No bootstrap required, opened a new log
I20260811 07:07:21.341791 17030 raft_consensus.cc:374] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:21.341813 17030 raft_consensus.cc:729] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15322815fc094adca693bd9b455d756e, State: Initialized, Role: FOLLOWER
I20260811 07:07:21.341814 17022 ts_tablet_manager.cc:1397] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:21.341868 17030 consensus_queue.cc:260] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.341915 17022 raft_consensus.cc:348] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.341974 17022 raft_consensus.cc:374] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:21.342000 17022 raft_consensus.cc:729] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac1b62c4f374119a380b0f741347d63, State: Initialized, Role: FOLLOWER
I20260811 07:07:21.342052 17022 consensus_queue.cc:260] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.342087 17030 ts_tablet_manager.cc:1428] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:21.342195 17022 ts_tablet_manager.cc:1428] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:21.342552 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.342300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:54631)
0811 07:07:21.342439 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.342509 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260811 07:07:21.342760 16364 heartbeater.cc:503] Master 127.3.236.188:44219 was elected leader, sending a full tablet report...
I20260811 07:07:21.342984 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.342881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:59737)
0811 07:07:21.342919 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.342957 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 07:07:21.343441 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.342393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:57581)
0811 07:07:21.343368 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.343408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":943,"thread_start_us":113,"threads_started":1}
I20260811 07:07:21.343639 16624 heartbeater.cc:503] Master 127.3.236.188:44219 was elected leader, sending a full tablet report...
I20260811 07:07:21.343837 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.343755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:55913)
0811 07:07:21.343787 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.343821 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260811 07:07:21.425560 17038 raft_consensus.cc:482] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:21.425683 17038 raft_consensus.cc:504] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.425899 17038 leader_election.cc:288] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375), 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:21.426126 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95235a0e04d2478084310cb45763e12f" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15322815fc094adca693bd9b455d756e" is_pre_election: true
I20260811 07:07:21.426174 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95235a0e04d2478084310cb45763e12f" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41758d2ff486493987ab4e49efa9b070" is_pre_election: true
I20260811 07:07:21.426249 16285 raft_consensus.cc:2454] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 0.
I20260811 07:07:21.426258 16776 raft_consensus.cc:2454] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 0.
I20260811 07:07:21.426435 16421 leader_election.cc:302] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41758d2ff486493987ab4e49efa9b070, 8ac1b62c4f374119a380b0f741347d63; no voters:
I20260811 07:07:21.426600 17038 raft_consensus.cc:2781] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:21.426676 17038 raft_consensus.cc:482] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:21.426707 17038 raft_consensus.cc:3037] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:21.427456 17038 raft_consensus.cc:504] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.427624 17038 leader_election.cc:288] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 election: Requested vote from peers 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375), 15322815fc094adca693bd9b455d756e (127.3.236.129:33919)
I20260811 07:07:21.427785 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95235a0e04d2478084310cb45763e12f" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41758d2ff486493987ab4e49efa9b070"
I20260811 07:07:21.427790 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95235a0e04d2478084310cb45763e12f" candidate_uuid: "8ac1b62c4f374119a380b0f741347d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15322815fc094adca693bd9b455d756e"
I20260811 07:07:21.427856 16776 raft_consensus.cc:3037] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:21.427877 16285 raft_consensus.cc:3037] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:21.428544 16285 raft_consensus.cc:2454] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 1.
I20260811 07:07:21.428563 16776 raft_consensus.cc:2454] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac1b62c4f374119a380b0f741347d63 in term 1.
I20260811 07:07:21.428725 16420 leader_election.cc:302] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15322815fc094adca693bd9b455d756e, 8ac1b62c4f374119a380b0f741347d63; no voters:
I20260811 07:07:21.428838 17038 raft_consensus.cc:2781] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:21.429035 17038 raft_consensus.cc:686] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac1b62c4f374119a380b0f741347d63, State: Running, Role: LEADER
I20260811 07:07:21.429105 17038 consensus_queue.cc:237] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } }
I20260811 07:07:21.429459 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.429318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:36779)
0811 07:07:21.429380 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.429414 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 07:07:21.429813 16059 catalog_manager.cc:5495] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "8ac1b62c4f374119a380b0f741347d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac1b62c4f374119a380b0f741347d63" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 35569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:21.452745 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.450344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.3.236.130:35569)
0811 07:07:21.450416 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:21.450418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:21.450426 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:21.450453 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:21.450453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:21.450464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:21.450540 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:21.450686 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.451183 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.451958 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:21.452183 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:21.452290 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:21.452440 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:21.452474 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:21.452537 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:21.452630 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:07:21.452656 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:21.453835 16776 raft_consensus.cc:1261] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Refusing update from remote peer 8ac1b62c4f374119a380b0f741347d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:21.453840 16285 raft_consensus.cc:1261] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Refusing update from remote peer 8ac1b62c4f374119a380b0f741347d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:21.454123 17038 consensus_queue.cc:1035] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15322815fc094adca693bd9b455d756e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:21.454241 17041 consensus_queue.cc:1035] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41758d2ff486493987ab4e49efa9b070" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:21.454891 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.454737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:33257)
0811 07:07:21.454794 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.454837 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 07:07:21.455526 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.455391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:39283)
0811 07:07:21.455466 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.455491 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 07:07:21.455907 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.455810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:41121)
0811 07:07:21.455846 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.455875 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 07:07:21.456739 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.456651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.129:57551)
0811 07:07:21.456687 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.456711 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 07:07:21.457873 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.457760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.131:38881)
0811 07:07:21.457816 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.457841 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:07:21.458065 17049 mvcc.cc:205] Tried to move back new op lower bound from 7317225641792602112 to 7317225641694126080. Current Snapshot: MvccSnapshot[applied={T|T < 7317225641792602112 or (T in {7317225641792602112})}]
I20260811 07:07:21.458575 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.458456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42057 (local address 127.3.236.130:43377)
0811 07:07:21.458497 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.458526 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:42057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 07:07:21.458731 16059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:21.458842 16059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:21.460122 16059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38490b4a056c406a939544918a5b6aa9: Sending DeleteTablet for 3 replicas of tablet 95235a0e04d2478084310cb45763e12f
I20260811 07:07:21.460433 16500 tablet_service.cc:1514] Processing DeleteTablet for tablet 95235a0e04d2478084310cb45763e12f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:21 UTC) from {username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:57456
I20260811 07:07:21.460474 16260 tablet_service.cc:1514] Processing DeleteTablet for tablet 95235a0e04d2478084310cb45763e12f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:21 UTC) from {username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:41174
I20260811 07:07:21.460477 16750 tablet_service.cc:1514] Processing DeleteTablet for tablet 95235a0e04d2478084310cb45763e12f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:21 UTC) from {username='oryx', principal='oryx/127.3.236.188@KRBTEST.COM'} at 127.0.0.1:49222
I20260811 07:07:21.460702 17064 tablet_replica.cc:330] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: stopping tablet replica
I20260811 07:07:21.460716 17061 tablet_replica.cc:330] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: stopping tablet replica
I20260811 07:07:21.460745 17057 tablet_replica.cc:330] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e: stopping tablet replica
I20260811 07:07:21.460831 17057 raft_consensus.cc:2229] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:21.460896 17057 raft_consensus.cc:2258] T 95235a0e04d2478084310cb45763e12f P 15322815fc094adca693bd9b455d756e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:21.460963 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 15976
I20260811 07:07:21.460768 17061 raft_consensus.cc:2229] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:21.461125 17061 raft_consensus.cc:2258] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:21.461469 17064 raft_consensus.cc:2229] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:21.461575 17064 raft_consensus.cc:2258] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:21.461679 17061 ts_tablet_manager.cc:1905] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:21.461908 17064 ts_tablet_manager.cc:1905] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:21.463500 17061 ts_tablet_manager.cc:1918] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:21.463542 17064 ts_tablet_manager.cc:1918] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:21.463574 17061 log.cc:1196] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/95235a0e04d2478084310cb45763e12f
I20260811 07:07:21.463594 17064 log.cc:1196] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/95235a0e04d2478084310cb45763e12f
I20260811 07:07:21.463825 17061 ts_tablet_manager.cc:1939] T 95235a0e04d2478084310cb45763e12f P 41758d2ff486493987ab4e49efa9b070: Deleting consensus metadata
I20260811 07:07:21.463829 17064 ts_tablet_manager.cc:1939] T 95235a0e04d2478084310cb45763e12f P 8ac1b62c4f374119a380b0f741347d63: Deleting consensus metadata
I20260811 07:07:21.464119 16033 catalog_manager.cc:4841] TS 41758d2ff486493987ab4e49efa9b070 (127.3.236.131:46375): tablet 95235a0e04d2478084310cb45763e12f (table test-table [id=b425147bf0ad466abcdac14446556e22]) successfully deleted
I20260811 07:07:21.464164 16031 catalog_manager.cc:4841] TS 8ac1b62c4f374119a380b0f741347d63 (127.3.236.130:35569): tablet 95235a0e04d2478084310cb45763e12f (table test-table [id=b425147bf0ad466abcdac14446556e22]) successfully deleted
I20260811 07:07:21.475118 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:21.474995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33919 (local address 127.0.0.1:41178)
0811 07:07:21.475036 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:21.475068 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 07:07:21.475256 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 16383
W20260811 07:07:21.475334 16031 catalog_manager.cc:4584] TS 15322815fc094adca693bd9b455d756e (127.3.236.129:33919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95235a0e04d2478084310cb45763e12f: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33919: connect: Connection refused (error 111)
I20260811 07:07:21.486608 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 16632
I20260811 07:07:21.504980 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 15472
I20260811 07:07:21.516004 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 15805
2026-08-11T07:07:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5066 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:21 dist-test-slave-2d19 krb5kdc[17070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:21 dist-test-slave-2d19 krb5kdc[17070](info): set up 2 sockets
Aug 11 07:07:21 dist-test-slave-2d19 krb5kdc[17070](info): commencing operation
krb5kdc: starting...
W20260811 07:07:22.813493 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:22 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:22Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab.
Entry for principal oryx/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab.
I20260811 07:07:22.944342 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34209
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab
--principal=oryx/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:34209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:23.029366 17092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:23.029562 17092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:23.029584 17092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:23.030718 17092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:23.030755 17092 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:07:23.030769 17092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:23.030782 17092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:23.030794 17092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:23.030807 17092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:23.032334 17092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:34209
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.190.keytab
--principal=oryx/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:23.032531 17092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:23.032711 17092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:23.037432 17103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.037475 17100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:23.037679 17092 server_base.cc:1034] running on GCE node
W20260811 07:07:23.037442 17101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:23.037936 17092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:23.038234 17092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:23.039404 17092 hybrid_clock.cc:648] HybridClock initialized: now 1786432043039401 us; error 39 us; skew 500 ppm
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:23.046311 17092 init.cc:377] Logged in from keytab as oryx/127.3.236.190@KRBTEST.COM (short username oryx)
I20260811 07:07:23.046818 17092 webserver.cc:460] Webserver started at http://127.3.236.190:39319/ using document root <none> and password file <none>
I20260811 07:07:23.047080 17092 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:23.047158 17092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:23.047308 17092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:23.048455 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "56ea45e3343042a7a81f9e6d2480682b"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "1e2edb83720465d22993a64565c4622a"
server_key_iv: "79036f7152127f1c000a60d924d6e0d2"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.048816 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "56ea45e3343042a7a81f9e6d2480682b"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "1e2edb83720465d22993a64565c4622a"
server_key_iv: "79036f7152127f1c000a60d924d6e0d2"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.050256 17092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:23.050957 17121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.051131 17092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:23.051240 17092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "56ea45e3343042a7a81f9e6d2480682b"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "1e2edb83720465d22993a64565c4622a"
server_key_iv: "79036f7152127f1c000a60d924d6e0d2"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.051316 17092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:23.073817 17092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:23.075778 17092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:23.075960 17092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:23.081318 17092 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:34209
I20260811 07:07:23.081373 17222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:34209 every 8 connection(s)
I20260811 07:07:23.081724 17092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:23.082410 17228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:23.084791 17228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Bootstrap starting.
I20260811 07:07:23.085464 17228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:23.085776 17228 log.cc:824] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:23.086560 17228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: No bootstrap required, opened a new log
I20260811 07:07:23.088825 17228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } }
I20260811 07:07:23.089007 17228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:23.089035 17228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ea45e3343042a7a81f9e6d2480682b, State: Initialized, Role: FOLLOWER
I20260811 07:07:23.089228 17228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } }
I20260811 07:07:23.089293 17228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:23.089339 17228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:23.089388 17228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:23.089906 17228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } }
I20260811 07:07:23.090111 17228 leader_election.cc:302] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56ea45e3343042a7a81f9e6d2480682b; no voters:
I20260811 07:07:23.090227 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 17092
I20260811 07:07:23.090305 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:23.090296 17228 leader_election.cc:288] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:23.090337 17235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:23.090484 17235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [term 1 LEADER]: Becoming Leader. State: Replica: 56ea45e3343042a7a81f9e6d2480682b, State: Running, Role: LEADER
I20260811 07:07:23.090502 4018 external_mini_cluster.cc:1428] Setting key 3404f1a9582e4ff803b98c6f4fee4800
I20260811 07:07:23.090562 17235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } }
I20260811 07:07:23.090598 17228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:23.090888 17235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56ea45e3343042a7a81f9e6d2480682b. Latest consensus state: current_term: 1 leader_uuid: "56ea45e3343042a7a81f9e6d2480682b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } } }
I20260811 07:07:23.090984 17235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:23.090902 17236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56ea45e3343042a7a81f9e6d2480682b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ea45e3343042a7a81f9e6d2480682b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34209 } } }
I20260811 07:07:23.091451 17236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:23.092124 17242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:23.092746 17242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:23.094309 17242 catalog_manager.cc:1347] Generated new cluster ID: eef43478bdc94d0788c82187a38b5797
I20260811 07:07:23.094364 17242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:23.100967 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.091057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.3.236.190:34209)
0811 07:07:23.091263 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.091273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.091305 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:07:23.091434 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.091441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.091502 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.092344 (+ 842us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.092641 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.093761 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.094681 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.095118 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.095233 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.097662 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.097666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.097667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.097679 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.099372 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.099752 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.099756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.099756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.099853 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.100086 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.100087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.100087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.100393 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.100581 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.100648 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.100786 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260811 07:07:23.103891 17242 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:23.104619 17242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:23.108455 17242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Generated new TSK 0
I20260811 07:07:23.108620 17242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
I20260811 07:07:23.140981 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34209
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:23.242738 17278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:23.243074 17278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:23.243162 17278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:23.244915 17278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:23.245025 17278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:23.245100 17278 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:07:23.245165 17278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:23.247543 17278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:23.247835 17278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:23.248128 17278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:23.253633 17285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.253659 17290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.253670 17288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:23.254254 17278 server_base.cc:1034] running on GCE node
I20260811 07:07:23.254503 17278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:23.254793 17278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:23.255991 17278 hybrid_clock.cc:648] HybridClock initialized: now 1786432043255973 us; error 37 us; skew 500 ppm
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:23.259265 17278 init.cc:377] Logged in from keytab as oryx/127.3.236.129@KRBTEST.COM (short username oryx)
I20260811 07:07:23.259711 17278 webserver.cc:460] Webserver started at http://127.3.236.129:42389/ using document root <none> and password file <none>
I20260811 07:07:23.259971 17278 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:23.260042 17278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:23.260205 17278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:23.261323 17278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "c8571613bf694c22a246bffd08e01e27"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "fc26d91fc9777186d171df3aefb0b3c9"
server_key_iv: "3e469c028da9867c566cb875d618b586"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.261703 17278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "c8571613bf694c22a246bffd08e01e27"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "fc26d91fc9777186d171df3aefb0b3c9"
server_key_iv: "3e469c028da9867c566cb875d618b586"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.263159 17278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:23.263967 17303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.264138 17278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:23.264241 17278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "c8571613bf694c22a246bffd08e01e27"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "fc26d91fc9777186d171df3aefb0b3c9"
server_key_iv: "3e469c028da9867c566cb875d618b586"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.264359 17278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:23.274225 17278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:23.276577 17278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:23.276736 17278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:23.277257 17278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:23.277323 17278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.277367 17278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:23.277413 17278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.284669 17278 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:41437
I20260811 07:07:23.284807 17501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:41437 every 8 connection(s)
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:23.285100 17278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:23.287279 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 17278
I20260811 07:07:23.287441 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:23.287655 4018 external_mini_cluster.cc:1428] Setting key d60cf335e35d5bacfb5bf510c59a99e3
I20260811 07:07:23.288650 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.277757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:35211 (local address 127.3.236.190:34209)
0811 07:07:23.277799 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.277801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.278042 (+ 241us) server_negotiation.cc:407] Connection header received
0811 07:07:23.278603 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.278604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.278620 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.278692 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.279650 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.280506 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.282179 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.282535 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.282610 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.285427 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.285429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.285429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.285438 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.286962 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.287441 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.287443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.287443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.287506 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.287823 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.287825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.287825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.287975 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.288049 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.288420 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.288534 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:07:23.288748 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.277746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.129:35211)
0811 07:07:23.277895 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.277930 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.278472 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.278810 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.278815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.279221 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.279539 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.279545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.280609 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.280610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.282079 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.282083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.282636 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.282637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.282787 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.283382 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.283395 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.285314 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.287045 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.287051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.287052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.287266 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.287618 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.287618 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.287619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.287750 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.288083 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.288090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.288245 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.288390 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.288563 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260811 07:07:23.289259 17502 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34209
I20260811 07:07:23.289366 17502 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:23.289559 17502 heartbeater.cc:511] Master 127.3.236.190:34209 requested a full tablet report, sending...
I20260811 07:07:23.290138 17143 ts_manager.cc:194] Registered new tserver with Master: c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437)
I20260811 07:07:23.290992 17143 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:35211
I20260811 07:07:23.298169 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.291429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:54701 (local address 127.3.236.190:34209)
0811 07:07:23.291468 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.291470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.291681 (+ 211us) server_negotiation.cc:407] Connection header received
0811 07:07:23.291790 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.291791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.291801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.291858 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.292351 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.292813 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.294003 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.294201 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.294266 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.295350 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.295351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.295352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.295358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.296736 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.297207 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.297209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.297209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.297266 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.297639 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.297640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.297641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.297749 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.297808 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.298035 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.298104 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 07:07:23.298357 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.291392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.129:54701)
0811 07:07:23.291581 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.291591 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.291716 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.291865 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.291866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.292106 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.292175 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.292178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.292928 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.292929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.293899 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.293903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.294290 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.294292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.294368 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.294799 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.294805 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.295247 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.296820 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.296822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.296822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.297060 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.297351 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.297352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.297353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.297545 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.297832 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.297834 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.297894 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.297986 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.298109 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":134,"threads_started":1}
WARNING: no policy specified for oryx/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
I20260811 07:07:23.318730 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34209
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:23.404577 17525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:23.404832 17525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:23.404872 17525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:23.406018 17525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:23.406104 17525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:23.406139 17525 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:07:23.406177 17525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:23.407692 17525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:23.407972 17525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:23.408222 17525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:23.413290 17540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.413323 17532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.413291 17534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:23.413499 17525 server_base.cc:1034] running on GCE node
I20260811 07:07:23.413751 17525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:23.414008 17525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:23.415184 17525 hybrid_clock.cc:648] HybridClock initialized: now 1786432043415165 us; error 37 us; skew 500 ppm
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:23.417865 17525 init.cc:377] Logged in from keytab as oryx/127.3.236.130@KRBTEST.COM (short username oryx)
I20260811 07:07:23.418376 17525 webserver.cc:460] Webserver started at http://127.3.236.130:41345/ using document root <none> and password file <none>
I20260811 07:07:23.418637 17525 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:23.418708 17525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:23.418869 17525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:23.419955 17525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "55ec2420776f449613d6d2d316c882b7"
server_key_iv: "6c449d01b9eb2ee91da5a39ebc2df0ed"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.420420 17525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "55ec2420776f449613d6d2d316c882b7"
server_key_iv: "6c449d01b9eb2ee91da5a39ebc2df0ed"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.421814 17525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:23.422595 17550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.422777 17525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:23.422873 17525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "55ec2420776f449613d6d2d316c882b7"
server_key_iv: "6c449d01b9eb2ee91da5a39ebc2df0ed"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.422962 17525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:23.433879 17525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:23.436220 17525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:23.436420 17525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:23.437000 17525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:23.437067 17525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.437108 17525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:23.437136 17525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:23.444978 17525 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:46425
I20260811 07:07:23.445019 17759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:46425 every 8 connection(s)
I20260811 07:07:23.445386 17525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:23.447328 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.437533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:40109 (local address 127.3.236.190:34209)
0811 07:07:23.437641 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.437644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.437807 (+ 163us) server_negotiation.cc:407] Connection header received
0811 07:07:23.438313 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.438315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.438330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.438415 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.439125 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.439967 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.441232 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.441593 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.441683 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.444448 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.444451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.444451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.444460 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.446184 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.446544 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.446545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.446545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.446596 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.446807 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.446808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.446808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.446909 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.446960 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.447200 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.447264 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 07:07:23.447587 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.437485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.130:40109)
0811 07:07:23.437665 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.437709 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.438215 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.438490 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.438494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.438821 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.439012 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.439017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.440092 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.440093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.441137 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.441140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.441700 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.441701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.441842 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.442353 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.442361 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.444348 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.446254 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.446260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.446261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.446471 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.446655 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.446656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.446656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.446752 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.446964 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.446968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.447087 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.447171 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.447325 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260811 07:07:23.448112 17760 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34209
I20260811 07:07:23.448201 17760 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:23.448410 17760 heartbeater.cc:511] Master 127.3.236.190:34209 requested a full tablet report, sending...
I20260811 07:07:23.448865 17144 ts_manager.cc:194] Registered new tserver with Master: 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425)
I20260811 07:07:23.449332 17144 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:40109
I20260811 07:07:23.454511 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 17525
I20260811 07:07:23.454633 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.449262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:45811 (local address 127.3.236.190:34209)
0811 07:07:23.449305 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.449307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.449451 (+ 144us) server_negotiation.cc:407] Connection header received
0811 07:07:23.449537 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.449538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.449549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.449635 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.450016 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.450520 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.451163 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.451333 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.451382 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.452345 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.452346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.452347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.452351 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.453490 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.453826 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.453827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.453827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.453896 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.454070 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.454071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.454071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.454201 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.454254 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.454430 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.454526 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:07:23.454633 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:23.454720 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.449216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.130:45811)
0811 07:07:23.449378 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.449391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.449478 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.449658 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.449659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.449874 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.449933 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.449935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.450609 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.450609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.451093 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.451094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.451398 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.451400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.451446 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.451921 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.451926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.452211 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.453590 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.453591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.453591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.453763 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.453964 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.453965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.453965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.454017 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.454259 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.454259 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.454328 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.454403 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.454512 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260811 07:07:23.454870 4018 external_mini_cluster.cc:1428] Setting key 7fc60e0a5d456ebc39fcf8f93ce2a89d
WARNING: no policy specified for oryx/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
I20260811 07:07:23.483815 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:34209
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:23.567813 17774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:23.568018 17774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:23.568043 17774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:23.569290 17774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:23.569334 17774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:23.569355 17774 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:07:23.569372 17774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:23.570804 17774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.236.190:34209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:23.570983 17774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:23.571162 17774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:23.576166 17783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.576162 17785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:23.576187 17780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:23.576323 17774 server_base.cc:1034] running on GCE node
I20260811 07:07:23.576642 17774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:23.576869 17774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:23.578011 17774 hybrid_clock.cc:648] HybridClock initialized: now 1786432043577999 us; error 26 us; skew 500 ppm
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:23.580534 17774 init.cc:377] Logged in from keytab as oryx/127.3.236.131@KRBTEST.COM (short username oryx)
I20260811 07:07:23.581024 17774 webserver.cc:460] Webserver started at http://127.3.236.131:38629/ using document root <none> and password file <none>
I20260811 07:07:23.581249 17774 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:23.581295 17774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:23.581463 17774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:23.582497 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "3c8df9f42a85232e766ccadbdfaac0d3"
server_key_iv: "5aa0454919116b2b5441a444742eec94"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.582864 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "3c8df9f42a85232e766ccadbdfaac0d3"
server_key_iv: "5aa0454919116b2b5441a444742eec94"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.584203 17774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:23.585014 17796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.585203 17774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:07:23.585278 17774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
format_stamp: "Formatted at 2026-08-11 07:07:23 on dist-test-slave-2d19"
server_key: "3c8df9f42a85232e766ccadbdfaac0d3"
server_key_iv: "5aa0454919116b2b5441a444742eec94"
server_key_version: "encryptionkey@0"
I20260811 07:07:23.585369 17774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:23.602144 17774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:23.603998 17774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:23.604161 17774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:23.604764 17774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:23.604806 17774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:23.604830 17774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:23.604842 17774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:23.612452 17774 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:43491
I20260811 07:07:23.612612 17974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:43491 every 8 connection(s)
I20260811 07:07:23.612907 17774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:23.614897 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.605191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52529 (local address 127.3.236.190:34209)
0811 07:07:23.605262 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.605265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.605454 (+ 189us) server_negotiation.cc:407] Connection header received
0811 07:07:23.605983 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.605985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.605999 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.606080 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.606766 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.607553 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.608901 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.609285 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.609358 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.612084 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.612086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.612087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.612100 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.613749 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.614109 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.614109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.614110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.614162 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.614373 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.614374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.614374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.614472 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.614526 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.614769 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.614835 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:23.615083 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.605167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.131:52529)
0811 07:07:23.605316 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.605356 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.605861 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.606146 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.606149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.606468 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.606668 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.606673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.607658 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.607659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.608767 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.608770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.609405 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.609406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.609534 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.610072 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.610086 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.611965 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.613823 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.613828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.613829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.614039 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.614222 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.614223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.614223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.614319 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.614534 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.614539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.614658 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.614740 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.614859 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260811 07:07:23.615556 17977 heartbeater.cc:347] Connected to a master server at 127.3.236.190:34209
I20260811 07:07:23.615640 17977 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:23.615823 17977 heartbeater.cc:511] Master 127.3.236.190:34209 requested a full tablet report, sending...
I20260811 07:07:23.616612 17143 ts_manager.cc:194] Registered new tserver with Master: a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:23.616986 17143 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:52529
I20260811 07:07:23.619509 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 17774
I20260811 07:07:23.619644 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:23.619856 4018 external_mini_cluster.cc:1428] Setting key 16a7d3de00af09045c46e0f1f580eaf9
I20260811 07:07:23.620795 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:07:23.622627 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.617115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:44149 (local address 127.3.236.190:34209)
0811 07:07:23.617173 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.617175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.617244 (+ 69us) server_negotiation.cc:407] Connection header received
0811 07:07:23.617307 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.617308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.617319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.617368 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.617750 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.618201 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.618848 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.619019 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.619082 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.620142 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.620144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.620144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.620151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.621551 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.621936 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.621936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.621937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.621985 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.622211 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.622212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.622212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.622302 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.622354 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.622517 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.622573 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:07:23.622777 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.616640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.3.236.131:44149)
0811 07:07:23.617196 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.617206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.617253 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.617388 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.617389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.617598 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:23.617677 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.617679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.618285 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.618286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.618784 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.618785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.619166 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.619167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.619228 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.619664 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:23.619670 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:23.620052 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:23.621634 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.621635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.621636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.621854 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.622048 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:23.622049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:23.622050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:23.622121 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:23.622369 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:23.622370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:23.622423 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.622502 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.622617 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":69,"threads_started":1}
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:07:23.644498 17245 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:23.638907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.3.236.190:34209)
0811 07:07:23.638969 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.638972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.639003 (+ 31us) server_negotiation.cc:407] Connection header received
0811 07:07:23.639040 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.639041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.639050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.639112 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.639704 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.640190 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.640987 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.641177 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.641254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.644380 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50724: BlockingRecv error: recv got EOF from 127.0.0.1:50724 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 07:07:23 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:23.652405 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.645399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.3.236.190:34209)
0811 07:07:23.645474 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.645477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.645550 (+ 73us) server_negotiation.cc:407] Connection header received
0811 07:07:23.645637 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.645638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.645648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.645714 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:23.646228 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.646703 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.647390 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.647639 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.647712 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.650001 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:23.650003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:23.650003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:23.650011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:23.651259 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.651592 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.651594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.651594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.651651 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:23.651857 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:23.651858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:23.651859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:23.651981 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:23.652086 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.652200 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.652300 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:07:23.653998 17143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:23.655609 17143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:23.662479 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41437 (local address 127.0.0.1:34254)
0811 07:07:23.658176 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.658216 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.658280 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.658551 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.658555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.658557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:23.658630 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.658633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.660060 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.660061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.661207 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.661210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.662140 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.662141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.662228 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.662246 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.662322 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.662382 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":107,"thread_start_us":60,"threads_started":1}
I20260811 07:07:23.662509 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.3.236.129:41437)
0811 07:07:23.658356 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.658361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.658372 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:23.658415 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.658417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.658433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.658532 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:23.658780 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.659945 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.661298 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.662054 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.662134 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.662229 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.662354 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.662393 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260811 07:07:23.662662 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.3.236.131:43491)
0811 07:07:23.658647 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.658653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.658851 (+ 198us) server_negotiation.cc:407] Connection header received
0811 07:07:23.658897 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.658899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.658916 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.658990 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:23.659176 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.659849 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.661476 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.662171 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.662241 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.662320 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.662521 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.662566 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
I20260811 07:07:23.662479 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:43491 (local address 127.0.0.1:45774)
0811 07:07:23.658643 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.658654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.658711 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.659057 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.659058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.659060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:23.659100 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.659101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.659937 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.659938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.661385 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.661387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.662247 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.662247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.662304 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.662308 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.662367 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.662395 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":109,"thread_start_us":81,"threads_started":1}
I20260811 07:07:23.663494 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46425 (local address 127.0.0.1:50046)
0811 07:07:23.658386 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.658395 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.658437 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.658750 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.658751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.658753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:23.658798 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.658800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.660139 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.660140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.661384 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.661387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.663338 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.663338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.663384 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.663387 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.663421 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.663449 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":68,"thread_start_us":71,"threads_started":1}
I20260811 07:07:23.663595 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.658169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.3.236.130:46425)
0811 07:07:23.658356 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.658361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.658543 (+ 182us) server_negotiation.cc:407] Connection header received
0811 07:07:23.658590 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.658592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.658608 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.658682 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:23.658881 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.659960 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.662363 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.663254 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.663330 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.663401 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.663443 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.663480 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260811 07:07:23.663581 17390 tablet_service.cc:1467] Processing CreateTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a (DEFAULT_TABLE table=test-table [id=48bfd9dfb4904445a49fe275021f1db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:23.663991 17390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e288fcc1ff43ff95c8d08270120a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:23.664238 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a (DEFAULT_TABLE table=test-table [id=48bfd9dfb4904445a49fe275021f1db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:23.664674 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e288fcc1ff43ff95c8d08270120a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:23.664690 17875 tablet_service.cc:1467] Processing CreateTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a (DEFAULT_TABLE table=test-table [id=48bfd9dfb4904445a49fe275021f1db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:23.665068 17875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e288fcc1ff43ff95c8d08270120a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:23.667935 18025 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Bootstrap starting.
I20260811 07:07:23.668277 18026 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Bootstrap starting.
I20260811 07:07:23.668774 18025 tablet_bootstrap.cc:654] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:23.668780 18028 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Bootstrap starting.
I20260811 07:07:23.669054 18026 tablet_bootstrap.cc:654] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:23.669314 18025 log.cc:824] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:23.669385 18028 tablet_bootstrap.cc:654] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:23.669488 18026 log.cc:824] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:23.669704 18028 log.cc:824] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:23.670258 18026 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: No bootstrap required, opened a new log
I20260811 07:07:23.670252 18025 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: No bootstrap required, opened a new log
I20260811 07:07:23.670331 18026 ts_tablet_manager.cc:1397] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:23.670331 18025 ts_tablet_manager.cc:1397] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:07:23.670506 18028 tablet_bootstrap.cc:492] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: No bootstrap required, opened a new log
I20260811 07:07:23.670586 18028 ts_tablet_manager.cc:1397] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:23.672145 18028 raft_consensus.cc:348] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672331 18028 raft_consensus.cc:374] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:23.672375 18028 raft_consensus.cc:729] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13ca7405e8f4b20bfaa3db891fef5ef, State: Initialized, Role: FOLLOWER
I20260811 07:07:23.672391 18025 raft_consensus.cc:348] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672395 18026 raft_consensus.cc:348] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672492 18028 consensus_queue.cc:260] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672559 18026 raft_consensus.cc:374] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:23.672559 18025 raft_consensus.cc:374] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:23.672598 18025 raft_consensus.cc:729] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Initialized, Role: FOLLOWER
I20260811 07:07:23.672598 18026 raft_consensus.cc:729] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Initialized, Role: FOLLOWER
I20260811 07:07:23.672708 18025 consensus_queue.cc:260] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672708 18026 consensus_queue.cc:260] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.672786 18028 ts_tablet_manager.cc:1428] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 07:07:23.672907 17977 heartbeater.cc:503] Master 127.3.236.190:34209 was elected leader, sending a full tablet report...
I20260811 07:07:23.672982 18026 ts_tablet_manager.cc:1428] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 07:07:23.672978 18025 ts_tablet_manager.cc:1428] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 07:07:23.673148 17502 heartbeater.cc:503] Master 127.3.236.190:34209 was elected leader, sending a full tablet report...
I20260811 07:07:23.673413 17760 heartbeater.cc:503] Master 127.3.236.190:34209 was elected leader, sending a full tablet report...
W20260811 07:07:23.695992 17762 tablet.cc:2369] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:07:23.785990 17506 tablet.cc:2369] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:07:23.863613 17978 tablet.cc:2369] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:07:23.995553 18034 raft_consensus.cc:482] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:23.995704 18034 raft_consensus.cc:504] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:23.996172 18034 leader_election.cc:288] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:23.999444 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.996419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46425 (local address 127.3.236.129:36781)
0811 07:07:23.996560 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.996572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.996655 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.996878 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.996879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.996882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:23.996964 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.996967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.997772 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.997773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.998546 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.998548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.999182 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.999183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.999254 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.999259 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.999311 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.999349 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260811 07:07:23.999475 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.996501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:36781 (local address 127.3.236.130:46425)
0811 07:07:23.996572 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.996579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.996660 (+ 81us) server_negotiation.cc:407] Connection header received
0811 07:07:23.996775 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.996777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.996792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.996858 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:23.997055 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.997646 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.998642 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.999093 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.999157 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.999165 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.999334 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.999379 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:07:23.999588 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.996782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:57495 (local address 127.3.236.131:43491)
0811 07:07:23.996856 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:23.996858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:23.996867 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:23.996895 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:23.996896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:23.996906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:23.996963 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:23.997151 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.997630 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.998775 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.999158 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.999231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.999239 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:23.999460 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 07:07:23.999506 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:07:23.999573 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:23.996632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:43491 (local address 127.3.236.129:57495)
0811 07:07:23.996755 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:23.996767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:23.996843 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:23.996972 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:23.996972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:23.996974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:23.997068 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.997070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.997724 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.997726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.998677 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:23.998680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:23.999254 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:23.999255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:23.999362 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:23.999367 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:07:23.999434 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:07:23.999476 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":83,"thread_start_us":48,"threads_started":1}
I20260811 07:07:24.000106 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e288fcc1ff43ff95c8d08270120a7a" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" is_pre_election: true
I20260811 07:07:24.000109 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e288fcc1ff43ff95c8d08270120a7a" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" is_pre_election: true
I20260811 07:07:24.000291 17674 raft_consensus.cc:2454] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.000290 17901 raft_consensus.cc:2454] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.000543 17308 leader_election.cc:302] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13ca7405e8f4b20bfaa3db891fef5ef, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.000717 18034 raft_consensus.cc:2781] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:24.000792 18034 raft_consensus.cc:482] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:24.000810 18034 raft_consensus.cc:3037] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.001637 18034 raft_consensus.cc:504] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.001787 18034 leader_election.cc:288] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Requested vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.001976 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e288fcc1ff43ff95c8d08270120a7a" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
I20260811 07:07:24.002007 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e288fcc1ff43ff95c8d08270120a7a" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
I20260811 07:07:24.002060 17674 raft_consensus.cc:3037] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.002070 17901 raft_consensus.cc:3037] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.002767 17674 raft_consensus.cc:2454] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.002818 17901 raft_consensus.cc:2454] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.003000 17308 leader_election.cc:302] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.003115 18034 raft_consensus.cc:2781] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:24.003264 18034 raft_consensus.cc:686] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Becoming Leader. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Running, Role: LEADER
I20260811 07:07:24.003372 18034 consensus_queue.cc:237] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.004190 17143 catalog_manager.cc:5495] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8571613bf694c22a246bffd08e01e27 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "c8571613bf694c22a246bffd08e01e27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:24.019869 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.017544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.3.236.129:41437)
0811 07:07:24.017609 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.017611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.017619 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:24.017677 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.017678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.017694 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.017753 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:24.017856 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.018333 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.019177 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.019360 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.019413 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.019566 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.019663 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.019720 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.019760 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.019777 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:24.021862 17901 raft_consensus.cc:1261] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.021862 17674 raft_consensus.cc:1261] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.022230 18034 consensus_queue.cc:1035] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.022450 18040 consensus_queue.cc:1035] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.026428 18050 mvcc.cc:205] Tried to move back new op lower bound from 7317225652310532096 to 7317225652238376960. Current Snapshot: MvccSnapshot[applied={T|T < 7317225652310532096 or (T in {7317225652310532096})}]
I20260811 07:07:24.028119 17143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.028204 17143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.029033 17143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Sending DeleteTablet for 3 replicas of tablet 49e288fcc1ff43ff95c8d08270120a7a
I20260811 07:07:24.029381 17390 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34254
I20260811 07:07:24.029639 18060 tablet_replica.cc:330] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: stopping tablet replica
I20260811 07:07:24.029633 17875 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45774
I20260811 07:07:24.029747 18060 raft_consensus.cc:2229] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:24.029882 18060 raft_consensus.cc:2258] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.029907 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e288fcc1ff43ff95c8d08270120a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50046
I20260811 07:07:24.030102 17143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:24.030246 17143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:24.030426 18060 ts_tablet_manager.cc:1905] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.030869 18061 tablet_replica.cc:330] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: stopping tablet replica
I20260811 07:07:24.031028 18061 raft_consensus.cc:2229] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.031481 18061 raft_consensus.cc:2258] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.032003 18061 ts_tablet_manager.cc:1905] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.032241 17875 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (DEFAULT_TABLE table=test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.032404 18060 ts_tablet_manager.cc:1918] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.032473 18060 log.cc:1196] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/49e288fcc1ff43ff95c8d08270120a7a
I20260811 07:07:24.032483 17875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.032732 18060 ts_tablet_manager.cc:1939] T 49e288fcc1ff43ff95c8d08270120a7a P c8571613bf694c22a246bffd08e01e27: Deleting consensus metadata
I20260811 07:07:24.033617 18061 ts_tablet_manager.cc:1918] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.033674 18061 log.cc:1196] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/49e288fcc1ff43ff95c8d08270120a7a
I20260811 07:07:24.033852 18061 ts_tablet_manager.cc:1939] T 49e288fcc1ff43ff95c8d08270120a7a P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting consensus metadata
I20260811 07:07:24.033982 18063 tablet_replica.cc:330] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: stopping tablet replica
I20260811 07:07:24.034109 18063 raft_consensus.cc:2229] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.034202 18063 raft_consensus.cc:2258] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.034219 18028 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Bootstrap starting.
I20260811 07:07:24.034632 18063 ts_tablet_manager.cc:1905] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.034670 18028 tablet_bootstrap.cc:654] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.035311 18028 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: No bootstrap required, opened a new log
I20260811 07:07:24.035337 17126 catalog_manager.cc:4841] TS a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491): tablet 49e288fcc1ff43ff95c8d08270120a7a (table test-table [id=48bfd9dfb4904445a49fe275021f1db4]) successfully deleted
I20260811 07:07:24.035365 18028 ts_tablet_manager.cc:1397] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:24.035531 18028 raft_consensus.cc:348] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.035607 18028 raft_consensus.cc:374] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.035629 18028 raft_consensus.cc:729] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13ca7405e8f4b20bfaa3db891fef5ef, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.035638 17128 catalog_manager.cc:4841] TS c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437): tablet 49e288fcc1ff43ff95c8d08270120a7a (table test-table [id=48bfd9dfb4904445a49fe275021f1db4]) successfully deleted
I20260811 07:07:24.035671 18028 consensus_queue.cc:260] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.035764 18028 ts_tablet_manager.cc:1428] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.035820 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (DEFAULT_TABLE table=test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.035960 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.036031 17390 tablet_service.cc:1467] Processing CreateTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (DEFAULT_TABLE table=test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.036137 17390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.036211 18063 ts_tablet_manager.cc:1918] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.036317 18063 log.cc:1196] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/49e288fcc1ff43ff95c8d08270120a7a
I20260811 07:07:24.036579 18063 ts_tablet_manager.cc:1939] T 49e288fcc1ff43ff95c8d08270120a7a P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting consensus metadata
I20260811 07:07:24.036962 17126 catalog_manager.cc:4841] TS 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425): tablet 49e288fcc1ff43ff95c8d08270120a7a (table test-table [id=48bfd9dfb4904445a49fe275021f1db4]) successfully deleted
I20260811 07:07:24.037360 18025 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Bootstrap starting.
I20260811 07:07:24.037448 18026 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Bootstrap starting.
I20260811 07:07:24.037771 18025 tablet_bootstrap.cc:654] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.037832 18026 tablet_bootstrap.cc:654] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.038455 18025 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: No bootstrap required, opened a new log
I20260811 07:07:24.038509 18025 ts_tablet_manager.cc:1397] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.038647 18025 raft_consensus.cc:348] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.038713 18025 raft_consensus.cc:374] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.038735 18025 raft_consensus.cc:729] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.038792 18025 consensus_queue.cc:260] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.039009 18026 tablet_bootstrap.cc:492] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: No bootstrap required, opened a new log
I20260811 07:07:24.039052 18026 ts_tablet_manager.cc:1397] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260811 07:07:24.039126 18025 ts_tablet_manager.cc:1428] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.039196 18026 raft_consensus.cc:348] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.039273 18026 raft_consensus.cc:374] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.039299 18026 raft_consensus.cc:729] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.039340 18026 consensus_queue.cc:260] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.039441 18026 ts_tablet_manager.cc:1428] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.157998 18040 raft_consensus.cc:482] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:24.158109 18040 raft_consensus.cc:504] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.158233 18040 leader_election.cc:288] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.158586 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2b8e0ad3ac474d91d7d14ad6da9a36" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" is_pre_election: true
I20260811 07:07:24.158576 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2b8e0ad3ac474d91d7d14ad6da9a36" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" is_pre_election: true
I20260811 07:07:24.158679 17901 raft_consensus.cc:2454] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.158700 17674 raft_consensus.cc:2454] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.158849 17308 leader_election.cc:302] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.158970 18040 raft_consensus.cc:2781] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:24.159018 18040 raft_consensus.cc:482] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:24.159042 18040 raft_consensus.cc:3037] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.159806 18040 raft_consensus.cc:504] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.159947 18040 leader_election.cc:288] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Requested vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.160194 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2b8e0ad3ac474d91d7d14ad6da9a36" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
I20260811 07:07:24.160218 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f2b8e0ad3ac474d91d7d14ad6da9a36" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
I20260811 07:07:24.160310 17674 raft_consensus.cc:3037] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.160342 17901 raft_consensus.cc:3037] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.160940 17674 raft_consensus.cc:2454] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.161064 17901 raft_consensus.cc:2454] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.161114 17308 leader_election.cc:302] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.161234 18040 raft_consensus.cc:2781] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:24.161288 18040 raft_consensus.cc:686] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Becoming Leader. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Running, Role: LEADER
I20260811 07:07:24.161334 18040 consensus_queue.cc:237] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.161959 17144 catalog_manager.cc:5495] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8571613bf694c22a246bffd08e01e27 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "c8571613bf694c22a246bffd08e01e27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:07:24 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432043, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.190@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:24.212057 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.205138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.3.236.190:34209)
0811 07:07:24.205315 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.205317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.205325 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:24.205348 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.205349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.205358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.205423 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:24.205868 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.206376 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.207028 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.207227 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.207277 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.209306 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:24.209307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:24.209308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:24.209315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:24.210801 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:24.211203 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:24.211204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:24.211204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:24.211265 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:24.211463 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:24.211464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:24.211465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:24.211577 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:24.211657 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.211814 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.211934 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260811 07:07:24.212117 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.205056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:34209 (local address 127.0.0.1:50736)
0811 07:07:24.205201 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.205213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.205289 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.205444 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.205445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.205750 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:24.205798 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.205800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.206445 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.206446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.206962 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.206963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.207292 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.207293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.207332 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.207834 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:24.207839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:24.209215 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:24.210901 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:24.210910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:24.210912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:24.211126 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:24.211338 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:24.211339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:24.211340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:24.211401 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:24.211647 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:24.211654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:24.211702 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.211791 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.211996 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":101,"thread_start_us":73,"threads_started":1}
I20260811 07:07:24.213899 17144 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50736:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:07:24.214111 17144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:24.215512 17390 tablet_service.cc:1467] Processing CreateTablet for tablet 216cd7b5f40b4bc2a7c8facd272c5762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=478a58466c9848c488f91a60376e098b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:07:24.215574 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 216cd7b5f40b4bc2a7c8facd272c5762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=478a58466c9848c488f91a60376e098b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:07:24.215692 17390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216cd7b5f40b4bc2a7c8facd272c5762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.215754 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216cd7b5f40b4bc2a7c8facd272c5762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.217127 18026 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: Bootstrap starting.
I20260811 07:07:24.217228 18025 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: Bootstrap starting.
I20260811 07:07:24.217309 17875 tablet_service.cc:1467] Processing CreateTablet for tablet 216cd7b5f40b4bc2a7c8facd272c5762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=478a58466c9848c488f91a60376e098b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:07:24.217445 17875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216cd7b5f40b4bc2a7c8facd272c5762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.217479 18026 tablet_bootstrap.cc:654] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.217633 18025 tablet_bootstrap.cc:654] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.218125 18026 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: No bootstrap required, opened a new log
I20260811 07:07:24.218171 18026 ts_tablet_manager.cc:1397] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.218209 18025 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: No bootstrap required, opened a new log
I20260811 07:07:24.218241 18025 ts_tablet_manager.cc:1397] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.218284 18026 raft_consensus.cc:348] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.218359 18026 raft_consensus.cc:374] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.218353 18025 raft_consensus.cc:348] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.218384 18026 raft_consensus.cc:729] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.218397 18025 raft_consensus.cc:374] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.218411 18025 raft_consensus.cc:729] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.218439 18026 consensus_queue.cc:260] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.218472 18025 consensus_queue.cc:260] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.218533 18026 ts_tablet_manager.cc:1428] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.218586 18025 ts_tablet_manager.cc:1428] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.218647 18033 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.218747 18033 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.219053 18028 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: Bootstrap starting.
I20260811 07:07:24.219228 18040 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.219314 18040 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.219483 18028 tablet_bootstrap.cc:654] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.220279 18028 tablet_bootstrap.cc:492] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: No bootstrap required, opened a new log
I20260811 07:07:24.220335 18028 ts_tablet_manager.cc:1397] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:24.220471 18028 raft_consensus.cc:348] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.220544 18028 raft_consensus.cc:374] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.220567 18028 raft_consensus.cc:729] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13ca7405e8f4b20bfaa3db891fef5ef, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.220625 18028 consensus_queue.cc:260] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.220721 18028 ts_tablet_manager.cc:1428] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.220780 18032 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.220898 18032 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.368279 18033 raft_consensus.cc:482] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:24.368417 18033 raft_consensus.cc:504] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.368832 18033 leader_election.cc:288] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.372066 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.369131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:59743 (local address 127.3.236.131:43491)
0811 07:07:24.369190 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.369192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.369401 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:07:24.369488 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.369489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.369498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.369593 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:24.369775 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.370246 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371309 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.371672 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371759 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.371768 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.371948 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.371990 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:07:24.372058 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.369099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:36841 (local address 127.3.236.129:41437)
0811 07:07:24.369154 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.369157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.369400 (+ 243us) server_negotiation.cc:407] Connection header received
0811 07:07:24.369473 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.369474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.369488 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.369558 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:24.369769 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.370245 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371298 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.371670 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371757 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.371766 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.371952 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.371989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:07:24.372084 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.369055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41437 (local address 127.3.236.130:36841)
0811 07:07:24.369317 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.369326 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.369410 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.369576 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.369577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.369579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:24.369695 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.369698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.370332 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.370333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.371203 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.371205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.371757 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.371840 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.371843 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.371938 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.371982 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":45,"thread_start_us":196,"threads_started":1}
I20260811 07:07:24.372088 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.369058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:43491 (local address 127.3.236.130:59743)
0811 07:07:24.369321 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.369329 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.369414 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.369602 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.369603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.369605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:24.369703 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.369705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.370343 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.370344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.371214 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.371216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.371760 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.371761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.371835 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.371842 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.371929 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.371974 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":33,"thread_start_us":191,"threads_started":1}
I20260811 07:07:24.372418 17434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "216cd7b5f40b4bc2a7c8facd272c5762" candidate_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8571613bf694c22a246bffd08e01e27" is_pre_election: true
I20260811 07:07:24.372572 17434 raft_consensus.cc:2454] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f1be8ff758c4fc79671d432c26ac3ee in term 0.
I20260811 07:07:24.372684 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "216cd7b5f40b4bc2a7c8facd272c5762" candidate_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" is_pre_election: true
I20260811 07:07:24.372787 17901 raft_consensus.cc:2454] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f1be8ff758c4fc79671d432c26ac3ee in term 0.
I20260811 07:07:24.372783 17557 leader_election.cc:302] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.372915 18033 raft_consensus.cc:2781] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:24.372965 18033 raft_consensus.cc:482] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:24.372988 18033 raft_consensus.cc:3037] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.373740 18033 raft_consensus.cc:504] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.373885 18033 leader_election.cc:288] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [CANDIDATE]: Term 1 election: Requested vote from peers c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.374046 17434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "216cd7b5f40b4bc2a7c8facd272c5762" candidate_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8571613bf694c22a246bffd08e01e27"
I20260811 07:07:24.374132 17434 raft_consensus.cc:3037] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.374118 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "216cd7b5f40b4bc2a7c8facd272c5762" candidate_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
I20260811 07:07:24.374220 17901 raft_consensus.cc:3037] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.374814 17434 raft_consensus.cc:2454] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f1be8ff758c4fc79671d432c26ac3ee in term 1.
I20260811 07:07:24.374832 17901 raft_consensus.cc:2454] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f1be8ff758c4fc79671d432c26ac3ee in term 1.
I20260811 07:07:24.375005 17557 leader_election.cc:302] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.375106 18033 raft_consensus.cc:2781] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:24.375281 18033 raft_consensus.cc:686] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 LEADER]: Becoming Leader. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Running, Role: LEADER
I20260811 07:07:24.375370 18033 consensus_queue.cc:237] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.375751 18095 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: TxnStatusTablet state changed. Reason: New leader 0f1be8ff758c4fc79671d432c26ac3ee. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.375852 18095 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:07:24.376076 18098 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:07:24.376195 18098 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:07:24.376116 17143 catalog_manager.cc:5495] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130). New cstate: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:24.402146 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.399283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46425 (local address 127.0.0.1:50054)
0811 07:07:24.399377 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.399391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.399445 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.399655 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.399656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.399658 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:07:24.399704 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.399706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.400629 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.400630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.401186 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.401187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.401652 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.401653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.401695 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.401710 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.401997 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.402004 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.402036 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.402072 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 07:07:24.402168 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.399406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.3.236.130:46425)
0811 07:07:24.399479 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.399481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.399507 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:07:24.399543 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.399544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.399565 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.399641 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:24.399752 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.400550 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.401270 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.401575 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.401647 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.401806 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.401932 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.402004 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.402049 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.402069 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:07:24.403357 17901 raft_consensus.cc:1261] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Refusing update from remote peer 0f1be8ff758c4fc79671d432c26ac3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.403421 17434 raft_consensus.cc:1261] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Refusing update from remote peer 0f1be8ff758c4fc79671d432c26ac3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.403657 18095 consensus_queue.cc:1035] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.403820 18033 consensus_queue.cc:1035] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.404865 18040 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: TxnStatusTablet state changed. Reason: New leader 0f1be8ff758c4fc79671d432c26ac3ee. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.404963 18040 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.405105 18032 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: TxnStatusTablet state changed. Reason: New leader 0f1be8ff758c4fc79671d432c26ac3ee. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.405193 18032 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.405963 18095 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.406049 18095 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:07:24.406210 18051 mvcc.cc:205] Tried to move back new op lower bound from 7317225653873750016 to 7317225653762101248. Current Snapshot: MvccSnapshot[applied={T|T < 7317225653873750016}]
I20260811 07:07:24.406423 18032 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.406519 18032 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P a13ca7405e8f4b20bfaa3db891fef5ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.406750 18033 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.406837 18033 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P 0f1be8ff758c4fc79671d432c26ac3ee: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:07:24.406864 18040 tablet_replica.cc:439] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } } }
I20260811 07:07:24.406939 18040 tablet_replica.cc:442] T 216cd7b5f40b4bc2a7c8facd272c5762 P c8571613bf694c22a246bffd08e01e27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:07:24.406904 18047 mvcc.cc:205] Tried to move back new op lower bound from 7317225653873750016 to 7317225653762101248. Current Snapshot: MvccSnapshot[applied={T|T < 7317225653873750016}]
I20260811 07:07:24.412298 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.409733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:56671 (local address 127.3.236.130:46425)
0811 07:07:24.409774 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.409776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.409916 (+ 140us) server_negotiation.cc:407] Connection header received
0811 07:07:24.409947 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.409947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.409957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.410019 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:24.410122 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.410825 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.411530 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.411827 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.411911 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.412047 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.412078 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.412131 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.412180 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.412197 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:24.412321 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.409673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46425 (local address 127.3.236.129:56671)
0811 07:07:24.409842 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.409852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.409911 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.410027 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.410028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.410029 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:07:24.410071 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.410072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.410894 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.410894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.411452 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.411454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.411909 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.411910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.411949 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.411965 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.412140 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.412142 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.412177 (+ 35us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.412214 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260811 07:07:24.414306 17901 raft_consensus.cc:1261] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.414340 17674 raft_consensus.cc:1261] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.414539 18040 consensus_queue.cc:1035] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.414657 18034 consensus_queue.cc:1035] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.426939 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.420845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:33057 (local address 127.3.236.129:41437)
0811 07:07:24.420891 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.420893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.424455 (+ 3562us) server_negotiation.cc:407] Connection header received
0811 07:07:24.424525 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.424525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.424536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.424607 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:24.424741 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.425274 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.426162 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.426347 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.426378 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.426587 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.426629 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.426720 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.426831 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.426859 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:24.427037 18115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.420786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41437 (local address 127.3.236.130:33057)
0811 07:07:24.424390 (+ 3604us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.424402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.424467 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.424597 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.424598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.424600 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:07:24.424676 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.424678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.425350 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.425351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.426029 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.426032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.426435 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.426435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.426480 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.426495 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.426725 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.426727 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.426822 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.426868 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3559,"thread_start_us":79,"threads_started":1}
I20260811 07:07:24.435720 17144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.435842 17144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.436524 17144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Sending DeleteTablet for 3 replicas of tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36
I20260811 07:07:24.436780 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50046
I20260811 07:07:24.436812 17875 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45774
I20260811 07:07:24.436812 17390 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34254
I20260811 07:07:24.436879 18063 tablet_replica.cc:330] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: stopping tablet replica
I20260811 07:07:24.436899 18060 tablet_replica.cc:330] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: stopping tablet replica
I20260811 07:07:24.436928 18061 tablet_replica.cc:330] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: stopping tablet replica
I20260811 07:07:24.436949 18063 raft_consensus.cc:2229] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.436951 18060 raft_consensus.cc:2229] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:24.436988 18063 raft_consensus.cc:2258] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.436988 18061 raft_consensus.cc:2229] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.437006 18060 raft_consensus.cc:2258] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.437029 18061 raft_consensus.cc:2258] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.437340 18060 ts_tablet_manager.cc:1905] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.437340 18061 ts_tablet_manager.cc:1905] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.437569 18063 ts_tablet_manager.cc:1905] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.439121 18063 ts_tablet_manager.cc:1918] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:07:24.439183 18063 log.cc:1196] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/0f2b8e0ad3ac474d91d7d14ad6da9a36
I20260811 07:07:24.439236 18061 ts_tablet_manager.cc:1918] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:07:24.439296 18061 log.cc:1196] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/0f2b8e0ad3ac474d91d7d14ad6da9a36
I20260811 07:07:24.439404 18063 ts_tablet_manager.cc:1939] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting consensus metadata
I20260811 07:07:24.439533 18061 ts_tablet_manager.cc:1939] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting consensus metadata
I20260811 07:07:24.439865 17126 catalog_manager.cc:4841] TS 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425): tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (table test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]) successfully deleted
I20260811 07:07:24.439955 17126 catalog_manager.cc:4841] TS a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491): tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (table test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]) successfully deleted
I20260811 07:07:24.442893 18060 ts_tablet_manager.cc:1918] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:07:24.442966 18060 log.cc:1196] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/0f2b8e0ad3ac474d91d7d14ad6da9a36
I20260811 07:07:24.443197 18060 ts_tablet_manager.cc:1939] T 0f2b8e0ad3ac474d91d7d14ad6da9a36 P c8571613bf694c22a246bffd08e01e27: Deleting consensus metadata
I20260811 07:07:24.443564 17128 catalog_manager.cc:4841] TS c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437): tablet 0f2b8e0ad3ac474d91d7d14ad6da9a36 (table test-table [id=2a16a588ee344d4aaf2ebfe59dd33d62]) successfully deleted
Aug 11 07:07:24 dist-test-slave-2d19 krb5kdc[17070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:07:24 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:24 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:07:24.466001 18079 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:24.460204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50738 (local address 127.3.236.190:34209)
0811 07:07:24.460354 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.460357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.460405 (+ 48us) server_negotiation.cc:407] Connection header received
0811 07:07:24.460488 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.460490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.460501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.460562 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:24.461154 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.461911 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.462665 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.462967 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.463057 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.465933 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50738: BlockingRecv error: recv got EOF from 127.0.0.1:50738 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 07:07:24 dist-test-slave-2d19 krb5kdc[17070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM
I20260811 07:07:24.475731 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.466891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.3.236.190:34209)
0811 07:07:24.466951 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.466953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.467021 (+ 68us) server_negotiation.cc:407] Connection header received
0811 07:07:24.467081 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.467082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.467092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.467146 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:24.467660 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.468440 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.469116 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.469388 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.469462 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.471682 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:24.471684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:24.471685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:24.471692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:24.473927 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:24.474390 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:24.474391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:24.474391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:24.474492 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:24.474773 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:24.474774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:24.474775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:24.474974 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:24.475155 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.475366 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.475625 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:07:24.478184 17144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:24.478463 17144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:24.483230 17390 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3739aa57f742c2ad267d1973741aeb (DEFAULT_TABLE table=test-table [id=462dcd825b2c46919782069147e04969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.483342 17875 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3739aa57f742c2ad267d1973741aeb (DEFAULT_TABLE table=test-table [id=462dcd825b2c46919782069147e04969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.483340 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3739aa57f742c2ad267d1973741aeb (DEFAULT_TABLE table=test-table [id=462dcd825b2c46919782069147e04969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.483384 17390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3739aa57f742c2ad267d1973741aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.483460 17875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3739aa57f742c2ad267d1973741aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.483485 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3739aa57f742c2ad267d1973741aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.484807 18026 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Bootstrap starting.
I20260811 07:07:24.485108 18026 tablet_bootstrap.cc:654] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.485194 18025 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Bootstrap starting.
I20260811 07:07:24.485335 18028 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Bootstrap starting.
I20260811 07:07:24.485594 18025 tablet_bootstrap.cc:654] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.485703 18028 tablet_bootstrap.cc:654] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.486219 18026 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: No bootstrap required, opened a new log
I20260811 07:07:24.486270 18026 ts_tablet_manager.cc:1397] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.486354 18026 raft_consensus.cc:348] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.486407 18026 raft_consensus.cc:374] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.486421 18026 raft_consensus.cc:729] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.486481 18026 consensus_queue.cc:260] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.486554 18025 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: No bootstrap required, opened a new log
I20260811 07:07:24.486600 18025 ts_tablet_manager.cc:1397] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.486601 18026 ts_tablet_manager.cc:1428] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.486748 18025 raft_consensus.cc:348] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.486810 18025 raft_consensus.cc:374] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.486831 18025 raft_consensus.cc:729] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.486876 18025 consensus_queue.cc:260] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.486959 18025 ts_tablet_manager.cc:1428] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.487035 18028 tablet_bootstrap.cc:492] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: No bootstrap required, opened a new log
I20260811 07:07:24.487082 18028 ts_tablet_manager.cc:1397] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:24.487263 18028 raft_consensus.cc:348] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.487329 18028 raft_consensus.cc:374] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.487349 18028 raft_consensus.cc:729] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13ca7405e8f4b20bfaa3db891fef5ef, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.487390 18028 consensus_queue.cc:260] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.487488 18028 ts_tablet_manager.cc:1428] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.770378 18040 raft_consensus.cc:482] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:24.770483 18040 raft_consensus.cc:504] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.770671 18040 leader_election.cc:288] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.770948 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3739aa57f742c2ad267d1973741aeb" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" is_pre_election: true
I20260811 07:07:24.770951 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3739aa57f742c2ad267d1973741aeb" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" is_pre_election: true
I20260811 07:07:24.771059 17674 raft_consensus.cc:2454] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.771086 17901 raft_consensus.cc:2454] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.771246 17308 leader_election.cc:302] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.771358 18040 raft_consensus.cc:2781] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:24.771401 18040 raft_consensus.cc:482] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:24.771430 18040 raft_consensus.cc:3037] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.772135 18040 raft_consensus.cc:504] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.772313 18040 leader_election.cc:288] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Requested vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.772472 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3739aa57f742c2ad267d1973741aeb" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
I20260811 07:07:24.772459 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3739aa57f742c2ad267d1973741aeb" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
I20260811 07:07:24.772547 17901 raft_consensus.cc:3037] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.772547 17674 raft_consensus.cc:3037] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.773311 17901 raft_consensus.cc:2454] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.773368 17674 raft_consensus.cc:2454] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.773528 17308 leader_election.cc:302] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13ca7405e8f4b20bfaa3db891fef5ef, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.773677 18040 raft_consensus.cc:2781] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:24.773787 18040 raft_consensus.cc:686] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Becoming Leader. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Running, Role: LEADER
I20260811 07:07:24.773844 18040 consensus_queue.cc:237] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.774452 17143 catalog_manager.cc:5495] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8571613bf694c22a246bffd08e01e27 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "c8571613bf694c22a246bffd08e01e27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:24.842839 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.840492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.3.236.129:41437)
0811 07:07:24.840564 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.840567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.840576 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:24.840638 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.840639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.840662 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.840721 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:24.840871 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.841341 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.842133 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.842331 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.842410 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.842563 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:24.842600 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:24.842654 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.842718 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.842742 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:24.843653 17901 raft_consensus.cc:1261] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.843683 17674 raft_consensus.cc:1261] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.843916 18113 consensus_queue.cc:1035] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.844054 18040 consensus_queue.cc:1035] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.846736 17141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.846835 17141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.847537 17141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Sending DeleteTablet for 3 replicas of tablet 0f3739aa57f742c2ad267d1973741aeb
I20260811 07:07:24.847791 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3739aa57f742c2ad267d1973741aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50046
I20260811 07:07:24.847818 17390 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3739aa57f742c2ad267d1973741aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34254
I20260811 07:07:24.847819 17875 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3739aa57f742c2ad267d1973741aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45774
I20260811 07:07:24.847890 18063 tablet_replica.cc:330] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: stopping tablet replica
I20260811 07:07:24.847908 18061 tablet_replica.cc:330] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: stopping tablet replica
I20260811 07:07:24.847931 18060 tablet_replica.cc:330] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: stopping tablet replica
I20260811 07:07:24.847945 18063 raft_consensus.cc:2229] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.847954 18061 raft_consensus.cc:2229] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.847972 18060 raft_consensus.cc:2229] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:24.847997 18063 raft_consensus.cc:2258] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.848002 18061 raft_consensus.cc:2258] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.848029 18060 raft_consensus.cc:2258] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.848030 17141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:24.848172 17141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:24.848510 18061 ts_tablet_manager.cc:1905] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.848510 18063 ts_tablet_manager.cc:1905] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.848640 18060 ts_tablet_manager.cc:1905] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.850335 18060 ts_tablet_manager.cc:1918] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.850338 18063 ts_tablet_manager.cc:1918] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.850395 18060 log.cc:1196] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/0f3739aa57f742c2ad267d1973741aeb
I20260811 07:07:24.850400 18063 log.cc:1196] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/0f3739aa57f742c2ad267d1973741aeb
I20260811 07:07:24.850502 18061 ts_tablet_manager.cc:1918] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:24.850556 18061 log.cc:1196] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/0f3739aa57f742c2ad267d1973741aeb
I20260811 07:07:24.850615 18060 ts_tablet_manager.cc:1939] T 0f3739aa57f742c2ad267d1973741aeb P c8571613bf694c22a246bffd08e01e27: Deleting consensus metadata
I20260811 07:07:24.850653 18063 ts_tablet_manager.cc:1939] T 0f3739aa57f742c2ad267d1973741aeb P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting consensus metadata
I20260811 07:07:24.850768 18061 ts_tablet_manager.cc:1939] T 0f3739aa57f742c2ad267d1973741aeb P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting consensus metadata
I20260811 07:07:24.850908 17128 catalog_manager.cc:4841] TS c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437): tablet 0f3739aa57f742c2ad267d1973741aeb (table test-table [id=462dcd825b2c46919782069147e04969]) successfully deleted
I20260811 07:07:24.850970 17126 catalog_manager.cc:4841] TS 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425): tablet 0f3739aa57f742c2ad267d1973741aeb (table test-table [id=462dcd825b2c46919782069147e04969]) successfully deleted
I20260811 07:07:24.851069 17126 catalog_manager.cc:4841] TS a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491): tablet 0f3739aa57f742c2ad267d1973741aeb (table test-table [id=462dcd825b2c46919782069147e04969]) successfully deleted
I20260811 07:07:24.851402 17875 tablet_service.cc:1467] Processing CreateTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 (DEFAULT_TABLE table=test-table [id=77294effa6bc48ac94f7b9b608daf3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.851380 17390 tablet_service.cc:1467] Processing CreateTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 (DEFAULT_TABLE table=test-table [id=77294effa6bc48ac94f7b9b608daf3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.851508 17390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e502e21fda8849d1bfeb93b149aa99d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.851507 17875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e502e21fda8849d1bfeb93b149aa99d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.852406 17631 tablet_service.cc:1467] Processing CreateTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 (DEFAULT_TABLE table=test-table [id=77294effa6bc48ac94f7b9b608daf3c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:24.852528 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e502e21fda8849d1bfeb93b149aa99d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:24.852699 18028 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Bootstrap starting.
I20260811 07:07:24.852787 18026 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: Bootstrap starting.
I20260811 07:07:24.853060 18026 tablet_bootstrap.cc:654] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.853123 18028 tablet_bootstrap.cc:654] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.853565 18026 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: No bootstrap required, opened a new log
I20260811 07:07:24.853615 18026 ts_tablet_manager.cc:1397] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:24.853714 18026 raft_consensus.cc:348] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.853787 18026 raft_consensus.cc:374] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.853818 18025 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Bootstrap starting.
I20260811 07:07:24.853832 18026 raft_consensus.cc:729] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.853901 18026 consensus_queue.cc:260] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.854020 18026 ts_tablet_manager.cc:1428] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.854156 18025 tablet_bootstrap.cc:654] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:24.854583 18028 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: No bootstrap required, opened a new log
I20260811 07:07:24.854635 18028 ts_tablet_manager.cc:1397] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:07:24.854784 18028 raft_consensus.cc:348] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.854878 18028 raft_consensus.cc:374] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.854918 18028 raft_consensus.cc:729] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13ca7405e8f4b20bfaa3db891fef5ef, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.854986 18028 consensus_queue.cc:260] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.855105 18028 ts_tablet_manager.cc:1428] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.855306 18025 tablet_bootstrap.cc:492] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: No bootstrap required, opened a new log
I20260811 07:07:24.855345 18025 ts_tablet_manager.cc:1397] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260811 07:07:24.855476 18025 raft_consensus.cc:348] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.855566 18025 raft_consensus.cc:374] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:24.855594 18025 raft_consensus.cc:729] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1be8ff758c4fc79671d432c26ac3ee, State: Initialized, Role: FOLLOWER
I20260811 07:07:24.855640 18025 consensus_queue.cc:260] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.855744 18025 ts_tablet_manager.cc:1428] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:24.903416 18032 raft_consensus.cc:482] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:24.903515 18032 raft_consensus.cc:504] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.903848 18032 leader_election.cc:288] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437), 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425)
I20260811 07:07:24.904353 18040 raft_consensus.cc:482] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:24.904443 18040 raft_consensus.cc:504] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.904577 18040 leader_election.cc:288] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.904842 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" is_pre_election: true
I20260811 07:07:24.904840 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" is_pre_election: true
I20260811 07:07:24.904932 17674 raft_consensus.cc:2454] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.904930 17901 raft_consensus.cc:2454] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 0.
I20260811 07:07:24.905087 17308 leader_election.cc:302] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13ca7405e8f4b20bfaa3db891fef5ef, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.905241 18040 raft_consensus.cc:2781] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:24.905301 18040 raft_consensus.cc:482] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:24.905331 18040 raft_consensus.cc:3037] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.906231 18040 raft_consensus.cc:504] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.906517 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef"
I20260811 07:07:24.906492 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "c8571613bf694c22a246bffd08e01e27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee"
I20260811 07:07:24.906601 17901 raft_consensus.cc:3037] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.906595 17674 raft_consensus.cc:3037] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:24.907300 17901 raft_consensus.cc:2454] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.907442 17674 raft_consensus.cc:2454] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8571613bf694c22a246bffd08e01e27 in term 1.
I20260811 07:07:24.907456 17308 leader_election.cc:302] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13ca7405e8f4b20bfaa3db891fef5ef, c8571613bf694c22a246bffd08e01e27; no voters:
I20260811 07:07:24.907545 18034 raft_consensus.cc:2781] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:24.907584 18034 raft_consensus.cc:686] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Becoming Leader. State: Replica: c8571613bf694c22a246bffd08e01e27, State: Running, Role: LEADER
I20260811 07:07:24.907660 18034 consensus_queue.cc:237] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } }
I20260811 07:07:24.907749 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.904104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:50123 (local address 127.3.236.129:41437)
0811 07:07:24.904158 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.904162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.904304 (+ 142us) server_negotiation.cc:407] Connection header received
0811 07:07:24.904390 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.904391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.904402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.904452 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:24.904674 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.905420 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.906640 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.907145 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.907197 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.907206 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.907629 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.907676 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:07:24.907805 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.904276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:55353 (local address 127.3.236.130:46425)
0811 07:07:24.904325 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:24.904327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:24.904338 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:24.904440 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:24.904441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:24.904452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:24.904504 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:24.904712 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.905419 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.906773 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.907367 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.907418 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.907427 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:24.907682 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 07:07:24.907725 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:24.907971 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.904000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41437 (local address 127.3.236.131:50123)
0811 07:07:24.904195 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.904214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.904324 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.904506 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.904507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.904508 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:24.904590 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.904592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.905521 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.905522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.906533 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.906536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.907448 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.907449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.907530 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.907535 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.907823 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.907881 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":65,"thread_start_us":56,"threads_started":1}
I20260811 07:07:24.908027 18040 leader_election.cc:288] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [CANDIDATE]: Term 1 election: Requested vote from peers 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425), a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491)
I20260811 07:07:24.908082 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.904033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46425 (local address 127.3.236.131:55353)
0811 07:07:24.904197 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.904217 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:24.904339 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:24.904547 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:24.904549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:24.904551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:24.904643 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.904645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.905509 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.905511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.906670 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:24.906673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:24.907509 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:24.907510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:24.907585 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:24.907590 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:07:24.907892 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 07:07:24.907945 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260811 07:07:24.908301 17434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8571613bf694c22a246bffd08e01e27" is_pre_election: true
I20260811 07:07:24.908499 17141 catalog_manager.cc:5495] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8571613bf694c22a246bffd08e01e27 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "c8571613bf694c22a246bffd08e01e27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8571613bf694c22a246bffd08e01e27" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 41437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 } health_report { overall_health: UNKNOWN } } }
I20260811 07:07:24.908605 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e502e21fda8849d1bfeb93b149aa99d2" candidate_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" is_pre_election: true
I20260811 07:07:24.908712 17674 raft_consensus.cc:2379] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a13ca7405e8f4b20bfaa3db891fef5ef in current term 1: Already voted for candidate c8571613bf694c22a246bffd08e01e27 in this term.
I20260811 07:07:24.908869 17800 leader_election.cc:302] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a13ca7405e8f4b20bfaa3db891fef5ef; no voters: 0f1be8ff758c4fc79671d432c26ac3ee, c8571613bf694c22a246bffd08e01e27
I20260811 07:07:24.909013 18032 raft_consensus.cc:2726] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 07:07:24.915311 17901 raft_consensus.cc:1261] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.915311 17674 raft_consensus.cc:1261] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Refusing update from remote peer c8571613bf694c22a246bffd08e01e27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:24.915519 18040 consensus_queue.cc:1035] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13ca7405e8f4b20bfaa3db891fef5ef" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 43491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.915632 18113 consensus_queue.cc:1035] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f1be8ff758c4fc79671d432c26ac3ee" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:24.925554 17141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.925648 17141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:24.926231 17141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56ea45e3343042a7a81f9e6d2480682b: Sending DeleteTablet for 3 replicas of tablet e502e21fda8849d1bfeb93b149aa99d2
I20260811 07:07:24.926496 17390 tablet_service.cc:1514] Processing DeleteTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34254
I20260811 07:07:24.926537 17875 tablet_service.cc:1514] Processing DeleteTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45774
I20260811 07:07:24.926543 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet e502e21fda8849d1bfeb93b149aa99d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:24 UTC) from {username='oryx', principal='oryx/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50046
I20260811 07:07:24.926602 18060 tablet_replica.cc:330] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: stopping tablet replica
I20260811 07:07:24.926616 18061 tablet_replica.cc:330] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: stopping tablet replica
I20260811 07:07:24.926642 18063 tablet_replica.cc:330] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: stopping tablet replica
I20260811 07:07:24.926656 18061 raft_consensus.cc:2229] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.926652 18060 raft_consensus.cc:2229] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:07:24.926689 18061 raft_consensus.cc:2258] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.926687 18063 raft_consensus.cc:2229] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:24.926718 18063 raft_consensus.cc:2258] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.926712 18060 raft_consensus.cc:2258] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:24.927004 18063 ts_tablet_manager.cc:1905] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.927119 18060 ts_tablet_manager.cc:1905] T e502e21fda8849d1bfeb93b149aa99d2 P c8571613bf694c22a246bffd08e01e27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.927256 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 17278
I20260811 07:07:24.930294 18063 ts_tablet_manager.cc:1918] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:07:24.930361 18063 log.cc:1196] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/e502e21fda8849d1bfeb93b149aa99d2
I20260811 07:07:24.930609 18063 ts_tablet_manager.cc:1939] T e502e21fda8849d1bfeb93b149aa99d2 P 0f1be8ff758c4fc79671d432c26ac3ee: Deleting consensus metadata
I20260811 07:07:24.936545 17126 catalog_manager.cc:4841] TS 0f1be8ff758c4fc79671d432c26ac3ee (127.3.236.130:46425): tablet e502e21fda8849d1bfeb93b149aa99d2 (table test-table [id=77294effa6bc48ac94f7b9b608daf3c1]) successfully deleted
I20260811 07:07:24.937239 18061 ts_tablet_manager.cc:1905] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:24.938762 18061 ts_tablet_manager.cc:1918] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:07:24.938851 18061 log.cc:1196] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/e502e21fda8849d1bfeb93b149aa99d2
I20260811 07:07:24.939062 18061 ts_tablet_manager.cc:1939] T e502e21fda8849d1bfeb93b149aa99d2 P a13ca7405e8f4b20bfaa3db891fef5ef: Deleting consensus metadata
I20260811 07:07:24.939452 17126 catalog_manager.cc:4841] TS a13ca7405e8f4b20bfaa3db891fef5ef (127.3.236.131:43491): tablet e502e21fda8849d1bfeb93b149aa99d2 (table test-table [id=77294effa6bc48ac94f7b9b608daf3c1]) successfully deleted
W20260811 07:07:24.945410 17128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:41437 (error 108)
I20260811 07:07:24.945724 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 17525
I20260811 07:07:24.951975 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:24.945782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:41437 (local address 127.0.0.1:34288)
0811 07:07:24.951793 (+ 6011us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:24.951909 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5982,"thread_start_us":56,"threads_started":1}
W20260811 07:07:24.952165 17128 catalog_manager.cc:4584] TS c8571613bf694c22a246bffd08e01e27 (127.3.236.129:41437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e502e21fda8849d1bfeb93b149aa99d2: Network error: Client connection negotiation failed: client connection to 127.3.236.129:41437: connect: Connection refused (error 111)
I20260811 07:07:24.959116 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 17774
I20260811 07:07:24.974742 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 17092
2026-08-11T07:07:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3455 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:25 dist-test-slave-2d19 krb5kdc[18174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:25 dist-test-slave-2d19 krb5kdc[18174](info): set up 2 sockets
Aug 11 07:07:25 dist-test-slave-2d19 krb5kdc[18174](info): commencing operation
krb5kdc: starting...
W20260811 07:07:26.328219 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:26 dist-test-slave-2d19 krb5kdc[18174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:26.455456 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36167
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:36167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:26.541564 18195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:26.541764 18195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:26.541787 18195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:26.542855 18195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:26.542891 18195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:26.542905 18195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:26.542917 18195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:26.542936 18195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:26.544443 18195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:36167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:26.544629 18195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:26.544806 18195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:26.549736 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:26.549765 18200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:26.549785 18204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:26.550077 18195 server_base.cc:1034] running on GCE node
I20260811 07:07:26.550415 18195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:26.550745 18195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:26.551940 18195 hybrid_clock.cc:648] HybridClock initialized: now 1786432046551917 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T07:07:26Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1573 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:26 dist-test-slave-2d19 krb5kdc[18210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:26 dist-test-slave-2d19 krb5kdc[18210](info): set up 2 sockets
Aug 11 07:07:26 dist-test-slave-2d19 krb5kdc[18210](info): commencing operation
krb5kdc: starting...
W20260811 07:07:27.860090 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:27 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:27.987954 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44805
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:44805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:28.073760 18228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.073978 18228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.074002 18228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.075177 18228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:28.075223 18228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.075237 18228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.075250 18228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:28.075263 18228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:28.076896 18228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:44805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.077183 18228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.077426 18228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:28.082798 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.082800 18235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.082847 18238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.083204 18228 server_base.cc:1034] running on GCE node
I20260811 07:07:28.083482 18228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:28.083801 18228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:28.084965 18228 hybrid_clock.cc:648] HybridClock initialized: now 1786432048084962 us; error 42 us; skew 500 ppm
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:28.092712 18228 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:28.093187 18228 webserver.cc:460] Webserver started at http://127.3.236.190:43797/ using document root <none> and password file <none>
I20260811 07:07:28.093417 18228 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:28.093463 18228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:28.093561 18228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:28.094666 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "4b5a7b52b81941c6aaf827e49dfac09b"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "924b5f37e58a0566e8411ccb4ea93179"
server_key_iv: "0f8dcde88a7c81406c2124ec8590b61c"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.094982 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "4b5a7b52b81941c6aaf827e49dfac09b"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "924b5f37e58a0566e8411ccb4ea93179"
server_key_iv: "0f8dcde88a7c81406c2124ec8590b61c"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.096413 18228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:07:28.097255 18249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.097435 18228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.097528 18228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "4b5a7b52b81941c6aaf827e49dfac09b"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "924b5f37e58a0566e8411ccb4ea93179"
server_key_iv: "0f8dcde88a7c81406c2124ec8590b61c"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.097589 18228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:28.114318 18228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:28.116341 18228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:28.116458 18228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:28.120426 18228 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:44805
I20260811 07:07:28.120499 18327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:44805 every 8 connection(s)
I20260811 07:07:28.120860 18228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:28.121804 18328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:28.123739 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 18228
I20260811 07:07:28.123829 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:28.124032 4018 external_mini_cluster.cc:1428] Setting key b861751dcfa02f4cc26b36e164831b53
I20260811 07:07:28.125074 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Bootstrap starting.
I20260811 07:07:28.125814 18328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:28.126199 18328 log.cc:824] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:28.127007 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: No bootstrap required, opened a new log
I20260811 07:07:28.129379 18328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.129565 18328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:28.129593 18328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5a7b52b81941c6aaf827e49dfac09b, State: Initialized, Role: FOLLOWER
I20260811 07:07:28.129761 18328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.129858 18328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:28.129914 18328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:28.129980 18328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:28.130553 18328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.130676 18328 leader_election.cc:302] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b5a7b52b81941c6aaf827e49dfac09b; no voters:
I20260811 07:07:28.130888 18328 leader_election.cc:288] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:28.130921 18335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:28.131045 18335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 LEADER]: Becoming Leader. State: Replica: 4b5a7b52b81941c6aaf827e49dfac09b, State: Running, Role: LEADER
I20260811 07:07:28.131134 18335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.131186 18328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:28.131489 18335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } } }
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:28.131675 18335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:28.131891 18335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b5a7b52b81941c6aaf827e49dfac09b. Latest consensus state: current_term: 1 leader_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } } }
I20260811 07:07:28.131970 18335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:28.132328 18351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:28.133227 18351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:28.134661 18351 catalog_manager.cc:1347] Generated new cluster ID: 3be53ce2fdf648d78a00d2382d13ef25
I20260811 07:07:28.134716 18351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:28.136272 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.124647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.3.236.190:44805)
0811 07:07:28.124847 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:28.124857 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:28.124892 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:07:28.125520 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:28.125530 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:28.125595 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:28.125754 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:28.126422 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.127777 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.128609 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.129304 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.129430 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.132141 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:28.132146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:28.132146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:28.132158 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:28.134272 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.134678 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.134681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.134682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.134800 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.135011 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.135012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.135012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.135313 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:28.135461 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:28.135903 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 07:07:28.136051 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260811 07:07:28.141570 18351 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:28.142081 18351 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:28.147761 18351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Generated new TSK 0
I20260811 07:07:28.147943 18351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:28.179719 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:28.264884 18369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.265100 18369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.265122 18369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.266381 18369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.266426 18369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.266453 18369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:28.267930 18369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.268185 18369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.268452 18369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:28.273586 18377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.273584 18380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.273696 18369 server_base.cc:1034] running on GCE node
W20260811 07:07:28.273597 18376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.274111 18369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:28.274374 18369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:28.275547 18369 hybrid_clock.cc:648] HybridClock initialized: now 1786432048275520 us; error 61 us; skew 500 ppm
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:28.278230 18369 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:28.278674 18369 webserver.cc:460] Webserver started at http://127.3.236.129:43119/ using document root <none> and password file <none>
I20260811 07:07:28.278926 18369 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:28.278998 18369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:28.279162 18369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:28.280201 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "23cd9a02770e42a7891261c50e061ff5"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "617aeb82b211d0ab3f897e5cde379b4b"
server_key_iv: "caa1cbe22e5eae1cfc1d99268afdc36b"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.280630 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "23cd9a02770e42a7891261c50e061ff5"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "617aeb82b211d0ab3f897e5cde379b4b"
server_key_iv: "caa1cbe22e5eae1cfc1d99268afdc36b"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.281966 18369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.282732 18388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.282930 18369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.283047 18369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "23cd9a02770e42a7891261c50e061ff5"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "617aeb82b211d0ab3f897e5cde379b4b"
server_key_iv: "caa1cbe22e5eae1cfc1d99268afdc36b"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.283125 18369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:28.290998 18369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:28.293273 18369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:28.293450 18369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:28.293668 18369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:28.293982 18369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:28.294051 18369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.294095 18369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:28.294122 18369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.299907 18369 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:37669
I20260811 07:07:28.299986 18562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:37669 every 8 connection(s)
I20260811 07:07:28.300367 18369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:28.305640 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 18369
I20260811 07:07:28.305765 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:28.305970 4018 external_mini_cluster.cc:1428] Setting key 4b50c1a8983bfa8115a35476f41db161
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:28.310256 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.301099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:57411 (local address 127.3.236.190:44805)
0811 07:07:28.301168 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:28.301170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:28.301394 (+ 224us) server_negotiation.cc:407] Connection header received
0811 07:07:28.301914 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:28.301915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:28.301934 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:28.301993 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:28.302912 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.303430 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.304451 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.304661 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.304718 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.306987 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:28.306988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:28.306989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:28.306997 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:28.308709 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.309251 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.309252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.309253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.309308 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.309598 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.309598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.309598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.309701 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:28.309753 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:28.310115 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 07:07:28.310190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:28.310420 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.301073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:57411)
0811 07:07:28.301261 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:28.301305 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:28.301792 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:28.302049 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:28.302052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:28.302625 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:28.302816 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.302821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.303514 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.303515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.304373 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.304376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.304738 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.304738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.304836 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.305309 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:28.305317 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:28.306890 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:28.308813 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.308822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.308824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.309169 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.309380 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.309381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.309382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.309529 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.309765 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:28.309771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:28.309946 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 07:07:28.310086 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 07:07:28.310214 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260811 07:07:28.311017 18563 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44805
I20260811 07:07:28.311115 18563 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:28.311313 18563 heartbeater.cc:511] Master 127.3.236.190:44805 requested a full tablet report, sending...
I20260811 07:07:28.311822 18269 ts_manager.cc:194] Registered new tserver with Master: 23cd9a02770e42a7891261c50e061ff5 (127.3.236.129:37669)
I20260811 07:07:28.312562 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:57411
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:28.335397 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:28.420241 18572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.420552 18572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.420591 18572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.421777 18572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.421845 18572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.421890 18572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:28.423367 18572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.423640 18572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.423882 18572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:28.429131 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.429186 18578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.429150 18577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.429291 18572 server_base.cc:1034] running on GCE node
I20260811 07:07:28.429682 18572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:28.429943 18572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:28.431145 18572 hybrid_clock.cc:648] HybridClock initialized: now 1786432048431140 us; error 42 us; skew 500 ppm
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:28.434124 18572 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:28.434568 18572 webserver.cc:460] Webserver started at http://127.3.236.130:44587/ using document root <none> and password file <none>
I20260811 07:07:28.434826 18572 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:28.434898 18572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:28.435055 18572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:28.436136 18572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "306aa1235dcc487197130aceacd94f5a"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "ef36d14828efdfee81e479792f34fb52"
server_key_iv: "1572d3e156df88f3b02b702964bb21d1"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.436571 18572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "306aa1235dcc487197130aceacd94f5a"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "ef36d14828efdfee81e479792f34fb52"
server_key_iv: "1572d3e156df88f3b02b702964bb21d1"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.437933 18572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.438742 18598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.438915 18572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.439043 18572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "306aa1235dcc487197130aceacd94f5a"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "ef36d14828efdfee81e479792f34fb52"
server_key_iv: "1572d3e156df88f3b02b702964bb21d1"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.439126 18572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:28.455452 18572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:28.457458 18572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:28.457639 18572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:28.457914 18572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:28.458246 18572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:28.458302 18572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.458348 18572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:28.458374 18572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.466275 18572 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41719
I20260811 07:07:28.466326 18772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41719 every 8 connection(s)
I20260811 07:07:28.466652 18572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:28.471534 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 18572
I20260811 07:07:28.471621 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:28.471891 4018 external_mini_cluster.cc:1428] Setting key c51cfb6202c5f5c4abce5353051ed178
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:28.476706 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.467379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:58131 (local address 127.3.236.190:44805)
0811 07:07:28.467469 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:28.467471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:28.467690 (+ 219us) server_negotiation.cc:407] Connection header received
0811 07:07:28.468195 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:28.468197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:28.468208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:28.468340 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:28.469310 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.469827 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.470842 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.471028 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.471087 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.473598 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:28.473600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:28.473600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:28.473608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:28.475101 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.475617 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.475617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.475618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.475670 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.475979 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.475981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.475981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.476105 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:28.476179 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:28.476524 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 07:07:28.476611 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:07:28.476915 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.467344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:58131)
0811 07:07:28.467555 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:28.467593 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:28.468073 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:28.468428 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:28.468432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:28.469021 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:28.469215 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.469219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.469923 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.469925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.470755 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.470757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.471114 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.471115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.471219 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.471844 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:28.471856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:28.473500 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:28.475188 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.475195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.475198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.475525 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.475739 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.475740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.475740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.475891 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.476182 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:28.476190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:28.476381 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 07:07:28.476489 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 07:07:28.476668 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260811 07:07:28.477449 18774 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44805
I20260811 07:07:28.477536 18774 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:28.477746 18774 heartbeater.cc:511] Master 127.3.236.190:44805 requested a full tablet report, sending...
I20260811 07:07:28.478173 18269 ts_manager.cc:194] Registered new tserver with Master: 306aa1235dcc487197130aceacd94f5a (127.3.236.130:41719)
I20260811 07:07:28.478731 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:58131
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:28.500818 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:28.583642 18784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.583850 18784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.583874 18784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.585000 18784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.585048 18784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.585075 18784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:28.586531 18784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.586783 18784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.587016 18784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:28.592278 18793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.592288 18792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.592320 18784 server_base.cc:1034] running on GCE node
W20260811 07:07:28.592294 18797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.592697 18784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:28.592967 18784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:28.594156 18784 hybrid_clock.cc:648] HybridClock initialized: now 1786432048594141 us; error 47 us; skew 500 ppm
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:28.596664 18784 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:28.597136 18784 webserver.cc:460] Webserver started at http://127.3.236.131:42705/ using document root <none> and password file <none>
I20260811 07:07:28.597378 18784 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:28.597446 18784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:28.597592 18784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:28.598651 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "55531ef68a0d48baab46c9806f2ec9a9"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "8a70e2f3733e86eb77555a9658b49e10"
server_key_iv: "a10d21e17099a5199c3d46b37bf511fa"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.599109 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "55531ef68a0d48baab46c9806f2ec9a9"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "8a70e2f3733e86eb77555a9658b49e10"
server_key_iv: "a10d21e17099a5199c3d46b37bf511fa"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.600540 18784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.601402 18808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.601608 18784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.601684 18784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "55531ef68a0d48baab46c9806f2ec9a9"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "8a70e2f3733e86eb77555a9658b49e10"
server_key_iv: "a10d21e17099a5199c3d46b37bf511fa"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.601778 18784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:28.623773 18784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:28.626019 18784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:28.626192 18784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:28.626451 18784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:28.626832 18784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:28.626881 18784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.626904 18784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:28.626946 18784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.633338 18784 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:41029
I20260811 07:07:28.633435 18998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:41029 every 8 connection(s)
I20260811 07:07:28.633733 18784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:28.636442 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 18784
I20260811 07:07:28.636559 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:28.636731 4018 external_mini_cluster.cc:1428] Setting key a05ac8d95914acc15d7f70bc729eb43a
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:28.643662 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.634437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:34733 (local address 127.3.236.190:44805)
0811 07:07:28.634506 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:28.634508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:28.634745 (+ 237us) server_negotiation.cc:407] Connection header received
0811 07:07:28.635314 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:28.635315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:28.635327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:28.635387 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:28.636557 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.637210 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.638202 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.638414 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.638502 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.640567 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:28.640569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:28.640569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:28.640576 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:28.641943 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.642480 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.642482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.642482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.642595 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:28.642995 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:28.642996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:28.642997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:28.643171 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:28.643219 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:28.643520 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 07:07:28.643592 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:28.643826 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:28.634436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:34733)
0811 07:07:28.634622 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:28.634659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:28.635206 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:28.635465 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:28.635468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:28.636176 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:28.636443 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.636449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.637307 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.637310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.638104 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:28.638106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:28.638518 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:28.638519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:28.638643 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:28.639073 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:28.639081 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:28.640469 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:28.642034 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.642047 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.642049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.642374 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.642712 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:28.642713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:28.642713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:28.642868 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:28.643270 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:28.643277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:28.643402 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 07:07:28.643484 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:07:28.643607 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260811 07:07:28.644371 19000 heartbeater.cc:347] Connected to a master server at 127.3.236.190:44805
I20260811 07:07:28.644472 19000 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:28.644667 19000 heartbeater.cc:511] Master 127.3.236.190:44805 requested a full tablet report, sending...
I20260811 07:07:28.645054 18269 ts_manager.cc:194] Registered new tserver with Master: 55531ef68a0d48baab46c9806f2ec9a9 (127.3.236.131:41029)
I20260811 07:07:28.645447 18269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:34733
I20260811 07:07:28.648523 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
Entry for principal oryx/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab.
WARNING: no policy specified for oryx/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
Entry for principal oryx/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab.
WARNING: no policy specified for oryx/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
Entry for principal oryx/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab.
I20260811 07:07:28.722805 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 18369
I20260811 07:07:28.743588 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 18572
I20260811 07:07:28.754629 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 18784
I20260811 07:07:28.767280 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 18228
I20260811 07:07:28.784511 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44805
--webserver_interface=127.3.236.190
--webserver_port=43797
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:44805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:28.868109 19015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.868387 19015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.868429 19015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.869589 19015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:28.869664 19015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.869686 19015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.869706 19015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:28.869738 19015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:28.871244 19015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:44805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:44805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.871541 19015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.871783 19015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:07:28.876906 19015 server_base.cc:1034] running on GCE node
W20260811 07:07:28.876883 19024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.876859 19028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:28.876869 19023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:28.877301 19015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:28.877599 19015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:28.878748 19015 hybrid_clock.cc:648] HybridClock initialized: now 1786432048878737 us; error 31 us; skew 500 ppm
Aug 11 07:07:28 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:28.881444 19015 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:28.881922 19015 webserver.cc:460] Webserver started at http://127.3.236.190:43797/ using document root <none> and password file <none>
I20260811 07:07:28.882179 19015 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:28.882259 19015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:28.883610 19015 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.884325 19037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:28.884498 19015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:28.884604 19015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "4b5a7b52b81941c6aaf827e49dfac09b"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "924b5f37e58a0566e8411ccb4ea93179"
server_key_iv: "0f8dcde88a7c81406c2124ec8590b61c"
server_key_version: "encryptionkey@0"
I20260811 07:07:28.884969 19015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:28.892993 19015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:28.895246 19015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:28.895416 19015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:28.899506 19015 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:44805
I20260811 07:07:28.899597 19118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:44805 every 8 connection(s)
I20260811 07:07:28.899971 19015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:28.901499 19119 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:07:28.902160 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Bootstrap starting.
I20260811 07:07:28.909931 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 19015
I20260811 07:07:28.910287 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:37669
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:44805
--webserver_port=43119
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:07:28.911298 19119 log.cc:824] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:28.913226 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:07:28.913516 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Bootstrap complete.
I20260811 07:07:28.915851 19119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.916074 19119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5a7b52b81941c6aaf827e49dfac09b, State: Initialized, Role: FOLLOWER
I20260811 07:07:28.916239 19119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.916347 19119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:28.916396 19119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:28.916445 19119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:07:28.917413 19119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.917500 19119 leader_election.cc:302] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b5a7b52b81941c6aaf827e49dfac09b; no voters:
I20260811 07:07:28.917728 19119 leader_election.cc:288] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:07:28.917770 19124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:07:28.917888 19124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [term 2 LEADER]: Becoming Leader. State: Replica: 4b5a7b52b81941c6aaf827e49dfac09b, State: Running, Role: LEADER
I20260811 07:07:28.918002 19119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:28.918062 19124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } }
I20260811 07:07:28.918447 19127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b5a7b52b81941c6aaf827e49dfac09b. Latest consensus state: current_term: 2 leader_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } } }
I20260811 07:07:28.918479 19129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5a7b52b81941c6aaf827e49dfac09b" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 44805 } } }
I20260811 07:07:28.918562 19127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:28.918566 19129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:28.918920 19139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:28.919478 19139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:28.919659 19139 catalog_manager.cc:1259] Loaded cluster ID: 3be53ce2fdf648d78a00d2382d13ef25
I20260811 07:07:28.919865 19139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:28.921031 19139 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:28.921200 19139 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b5a7b52b81941c6aaf827e49dfac09b: Loaded TSK: 0
I20260811 07:07:28.921403 19139 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:07:28.995373 19123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:28.995579 19123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:28.995604 19123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:28.996855 19123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:28.996899 19123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:28.996927 19123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:28.998374 19123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.129.keytab
--principal=oryx/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:37669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=43119
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:28.998610 19123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:28.998870 19123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:29.005046 19167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.005053 19170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.005059 19168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:29.005321 19123 server_base.cc:1034] running on GCE node
I20260811 07:07:29.005565 19123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:29.005841 19123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:29.007030 19123 hybrid_clock.cc:648] HybridClock initialized: now 1786432049007002 us; error 40 us; skew 500 ppm
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432049, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:29.009629 19123 init.cc:377] Logged in from keytab as oryx/127.3.236.129@KRBTEST.COM (short username oryx)
I20260811 07:07:29.010142 19123 webserver.cc:460] Webserver started at http://127.3.236.129:43119/ using document root <none> and password file <none>
I20260811 07:07:29.010407 19123 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:29.010468 19123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:29.011898 19123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:29.012549 19181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.012710 19123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:07:29.012820 19123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "23cd9a02770e42a7891261c50e061ff5"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "617aeb82b211d0ab3f897e5cde379b4b"
server_key_iv: "caa1cbe22e5eae1cfc1d99268afdc36b"
server_key_version: "encryptionkey@0"
I20260811 07:07:29.013166 19123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:29.024724 19123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:29.026990 19123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:29.027134 19123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:29.027416 19123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:29.027809 19123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:29.027848 19123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.027870 19123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:29.027884 19123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.035768 19123 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:37669
I20260811 07:07:29.035847 19365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:37669 every 8 connection(s)
I20260811 07:07:29.036240 19123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:29.036489 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 19123
I20260811 07:07:29.036799 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:41719
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:44805
--webserver_port=44587
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.046875 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.037223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:42625)
0811 07:07:29.037441 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.037485 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.037967 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.038515 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.038528 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.039272 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.039470 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.039475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.040803 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.040804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.042053 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.042055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.042950 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.042951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.043094 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.043737 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.043748 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.046672 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
W20260811 07:07:29.047078 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.037252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:42625 (local address 127.3.236.190:44805)
0811 07:07:29.038138 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.038145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.038172 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:07:29.038271 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.038275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.038315 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.038461 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.039607 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.040691 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.042150 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.042845 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.042978 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.046856 (+ 3878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:42625: BlockingRecv error: recv got EOF from 127.3.236.129:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":79,"threads_started":1}
W20260811 07:07:29.047569 19367 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:44805 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.236.190:44805: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.052707 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.047810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:34579)
0811 07:07:29.047856 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.047866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.047910 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.048093 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.048094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.048277 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.048337 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.048339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.049340 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.049341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.049841 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.049842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.050327 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.050327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.050377 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.050839 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.050844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.052658 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 07:07:29.052835 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.047873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:34579 (local address 127.3.236.190:44805)
0811 07:07:29.047920 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.047921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.047933 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:29.047982 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.047983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.047996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.048066 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.048414 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.049236 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.049924 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.050245 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.050322 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.052772 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:34579: BlockingRecv error: recv got EOF from 127.3.236.129:34579 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.057973 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.052967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:38357)
0811 07:07:29.053149 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.053160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.053221 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.053337 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.053337 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.053470 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.053522 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.053523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.054430 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.054430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.054996 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.054997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.055459 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.055460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.055504 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.055991 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.055995 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.057886 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":129,"threads_started":1}
W20260811 07:07:29.058149 19367 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:07:29.058291 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.053037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:38357 (local address 127.3.236.190:44805)
0811 07:07:29.053067 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.053069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.053232 (+ 163us) server_negotiation.cc:407] Connection header received
0811 07:07:29.053257 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.053258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.053269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.053329 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.053599 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.054346 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.055074 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.055372 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.055437 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.058240 (+ 2803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:38357: BlockingRecv error: recv got EOF from 127.3.236.129:38357 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260811 07:07:29.126752 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:29.126948 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:29.126969 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:29.128126 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:29.128170 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:29.128221 19371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:29.129653 19371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.130.keytab
--principal=oryx/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:41719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=44587
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:29.129915 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:29.130159 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:29.135360 19386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.135360 19382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.135412 19384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:29.135672 19371 server_base.cc:1034] running on GCE node
I20260811 07:07:29.135881 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:29.136129 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:29.137297 19371 hybrid_clock.cc:648] HybridClock initialized: now 1786432049137285 us; error 27 us; skew 500 ppm
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432049, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:29.139788 19371 init.cc:377] Logged in from keytab as oryx/127.3.236.130@KRBTEST.COM (short username oryx)
I20260811 07:07:29.140301 19371 webserver.cc:460] Webserver started at http://127.3.236.130:44587/ using document root <none> and password file <none>
I20260811 07:07:29.140565 19371 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:29.140648 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:29.142004 19371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:29.142597 19397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.142771 19371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:29.142877 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "306aa1235dcc487197130aceacd94f5a"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "ef36d14828efdfee81e479792f34fb52"
server_key_iv: "1572d3e156df88f3b02b702964bb21d1"
server_key_version: "encryptionkey@0"
I20260811 07:07:29.143256 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:29.154750 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:29.156771 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:29.156939 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:29.157183 19371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:29.157526 19371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:29.157586 19371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.157629 19371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:29.157657 19371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.163674 19371 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41719
I20260811 07:07:29.163758 19597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41719 every 8 connection(s)
I20260811 07:07:29.164099 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:07:29.171286 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.164916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:46377 (local address 127.3.236.190:44805)
0811 07:07:29.164986 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.164988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.165192 (+ 204us) server_negotiation.cc:407] Connection header received
0811 07:07:29.165693 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.165695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.165710 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.165762 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.166673 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.167159 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.168164 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.168372 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.168426 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.171241 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:46377: BlockingRecv error: recv got EOF from 127.3.236.130:46377 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:07:29.171291 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.164895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:46377)
0811 07:07:29.165062 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.165098 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.165596 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.165816 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.165819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.166380 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.166584 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.166588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.167269 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.167271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.168075 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.168077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.168443 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.168444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.168558 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.168998 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.169007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.171150 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
W20260811 07:07:29.171516 19601 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:44805 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.236.190:44805: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
I20260811 07:07:29.173723 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 19371
I20260811 07:07:29.174062 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:41029
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:44805
--webserver_port=42705
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.176115 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.171753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:48405)
0811 07:07:29.171805 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.171816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.171862 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.172003 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.172003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.172111 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.172155 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.172156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.172733 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.172734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.173277 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.173278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.173628 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.173629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.173674 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.174139 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.174145 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.176013 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 07:07:29.176191 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.171839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:48405 (local address 127.3.236.190:44805)
0811 07:07:29.171906 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.171908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.171923 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:07:29.171942 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.171942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.171953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.171998 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.172213 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.172667 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.173344 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.173554 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.173626 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.176116 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:48405: BlockingRecv error: recv got EOF from 127.3.236.130:48405 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.180617 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.176602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:45275)
0811 07:07:29.176658 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.176669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.176731 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.176875 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.176876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.177003 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.177034 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.177036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.177631 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.177632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.178120 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.178121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.178465 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.178466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.178518 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.178908 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.178913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.180575 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 07:07:29.180749 19601 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:07:29.180768 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.176678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:45275 (local address 127.3.236.190:44805)
0811 07:07:29.176728 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.176730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.176738 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:29.176793 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.176794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.176802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.176871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.177109 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.177549 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.178203 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.178382 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.178454 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.180690 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:45275: BlockingRecv error: recv got EOF from 127.3.236.130:45275 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 07:07:29.262813 19609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:29.263028 19609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:29.263052 19609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:29.264143 19609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:29.264185 19609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:29.264211 19609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:29.265655 19609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/krb5kdc/oryx_127.3.236.131.keytab
--principal=oryx/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:41029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=42705
--tserver_master_addrs=127.3.236.190:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:29.265839 19609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:29.266029 19609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:29.271157 19622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.271131 19618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:29.271132 19620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:29.271166 19609 server_base.cc:1034] running on GCE node
I20260811 07:07:29.271534 19609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:29.271728 19609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:29.272914 19609 hybrid_clock.cc:648] HybridClock initialized: now 1786432049272906 us; error 41 us; skew 500 ppm
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432049, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:29.275275 19609 init.cc:377] Logged in from keytab as oryx/127.3.236.131@KRBTEST.COM (short username oryx)
I20260811 07:07:29.275736 19609 webserver.cc:460] Webserver started at http://127.3.236.131:42705/ using document root <none> and password file <none>
I20260811 07:07:29.275923 19609 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:29.275974 19609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:29.277338 19609 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:29.277882 19638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.278081 19609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:29.278154 19609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "55531ef68a0d48baab46c9806f2ec9a9"
format_stamp: "Formatted at 2026-08-11 07:07:28 on dist-test-slave-2d19"
server_key: "8a70e2f3733e86eb77555a9658b49e10"
server_key_iv: "a10d21e17099a5199c3d46b37bf511fa"
server_key_version: "encryptionkey@0"
I20260811 07:07:29.278524 19609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:29.313910 19609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:29.321249 19609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:29.321506 19609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:29.322232 19609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:29.323429 19609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:29.323485 19609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.323529 19609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:29.323553 19609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:29.346522 19609 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:41029
I20260811 07:07:29.346622 19855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:41029 every 8 connection(s)
I20260811 07:07:29.347358 19609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:29.351792 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 19609
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:07:29.365298 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.348572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:42077 (local address 127.3.236.190:44805)
0811 07:07:29.348690 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.348692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.349097 (+ 405us) server_negotiation.cc:407] Connection header received
0811 07:07:29.350176 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.350179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.350192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.350318 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.353087 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.354030 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.358049 (+ 4019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.358701 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.359208 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.365211 (+ 6003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:42077: BlockingRecv error: recv got EOF from 127.3.236.131:42077 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:07:29.365454 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.348641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:42077)
0811 07:07:29.348904 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.348960 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.350002 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.350448 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.350454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.351994 (+ 1540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.352816 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.352831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.354225 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.354226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.357841 (+ 3615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.357845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.359142 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.359144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.359437 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.360671 (+ 1234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.360687 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.365113 (+ 4426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":123,"threads_started":1}
W20260811 07:07:29.366076 19856 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:44805 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.236.190:44805: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
I20260811 07:07:29.369274 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.352868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.3.236.190:44805)
0811 07:07:29.353124 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.353127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.353200 (+ 73us) server_negotiation.cc:407] Connection header received
0811 07:07:29.353236 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.353236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.353254 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.353340 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.354616 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.356482 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.359621 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.360174 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.360370 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.362908 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:29.362913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:29.362914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:29.362931 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:29.366423 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:29.367389 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:29.367392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:29.367392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:29.367555 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:29.367920 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:29.367926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:29.367926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:29.368533 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:29.368873 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:29.369024 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 07:07:29.369159 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.374090 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.366508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:43893)
0811 07:07:29.366674 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.366697 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.366778 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.367118 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.367120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.367370 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.367451 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.367453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.368910 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.368911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.369810 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.369813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.370477 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.370479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.370598 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.371385 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.371391 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.374025 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 07:07:29.374396 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.366582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:43893 (local address 127.3.236.190:44805)
0811 07:07:29.366715 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.366717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.366751 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:07:29.366870 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.366871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.366886 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.366958 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.367533 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.368821 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.369912 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.370334 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.370456 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.374190 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:43893: BlockingRecv error: recv got EOF from 127.3.236.131:43893 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:29 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:29.379899 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:29.374627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:38651)
0811 07:07:29.374692 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:29.374709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:29.374786 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:29.375018 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:29.375019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:29.375228 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:29.375263 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.375264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.375975 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.375976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.376866 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:29.376871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.377268 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.377270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:29.377318 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.377743 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:29.377747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:29.379837 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 07:07:29.379977 19370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:29.374687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38651 (local address 127.3.236.190:44805)
0811 07:07:29.374765 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:29.374767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:29.374779 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:07:29.374923 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:29.374924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:29.374933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:29.374998 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:29.375331 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.375906 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.376946 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:29.377191 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:29.377267 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:29.379935 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:38651: BlockingRecv error: recv got EOF from 127.3.236.131:38651 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 07:07:29.380200 19856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:30.068189 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:30.058771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:54127)
0811 07:07:30.059002 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:30.059014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:30.059139 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:30.059385 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:30.059392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:30.059790 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:30.059934 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.059937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.061165 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.061168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.062473 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.062477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.063642 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.063652 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.063876 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.064839 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:30.064844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:30.068057 (+ 3213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":133,"threads_started":1}
W20260811 07:07:30.068427 19870 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:30.058920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:54127 (local address 127.3.236.190:44805)
0811 07:07:30.059148 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:30.059150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:30.059159 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:30.059212 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:30.059212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:30.059231 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:30.059345 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:30.060048 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.060991 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.062742 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.063260 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.063623 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.068202 (+ 4579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:54127: BlockingRecv error: recv got EOF from 127.3.236.129:54127 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":152,"threads_started":1}
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:30.190179 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:30.181281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:54365)
0811 07:07:30.181462 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:30.181477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:30.181571 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:30.181884 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:30.181885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:30.182218 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:30.182397 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.182400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.183562 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.183563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.185038 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.185043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.185529 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.185536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.185690 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.186397 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:30.186402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:30.189971 (+ 3569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
W20260811 07:07:30.190264 19870 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:30.181385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:54365 (local address 127.3.236.190:44805)
0811 07:07:30.181466 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:30.181468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:30.181563 (+ 95us) server_negotiation.cc:407] Connection header received
0811 07:07:30.181654 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:30.181655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:30.181676 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:30.181830 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:30.182560 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.183366 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.185129 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.185391 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.185546 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.190162 (+ 4616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:54365: BlockingRecv error: recv got EOF from 127.3.236.130:54365 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:30 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:30.385782 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:30.380844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:50053)
0811 07:07:30.380965 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:30.380980 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:30.381047 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:30.381251 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:30.381252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:30.381482 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:30.381534 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.381536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.382194 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.382195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.382857 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:30.382860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.383209 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.383209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:30.383251 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.383769 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:30.383774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:30.385712 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
W20260811 07:07:30.385998 19870 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:30.380878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:50053 (local address 127.3.236.190:44805)
0811 07:07:30.380980 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:30.380982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:30.381056 (+ 74us) server_negotiation.cc:407] Connection header received
0811 07:07:30.381131 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:30.381132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:30.381150 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:30.381235 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:30.381619 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.382119 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.382948 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:30.383135 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:30.383195 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:30.385921 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:50053: BlockingRecv error: recv got EOF from 127.3.236.131:50053 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:31.074088 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:31.069170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:52751)
0811 07:07:31.069325 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:31.069336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:31.069391 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:31.069607 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:31.069608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:31.069787 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:31.069838 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.069839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.070540 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.070541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.071105 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.071107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.071471 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.071471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.071518 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.072090 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:31.072094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:31.074028 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260811 07:07:31.074231 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:31.069227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:52751 (local address 127.3.236.190:44805)
0811 07:07:31.069445 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:31.069447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:31.069456 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:31.069479 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:31.069479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:31.069491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:31.069556 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:31.069942 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.070461 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.071191 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.071400 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.071458 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.074139 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:52751: BlockingRecv error: recv got EOF from 127.3.236.129:52751 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":159,"threads_started":1}
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:31.195528 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:31.190806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:50569)
0811 07:07:31.190934 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:31.190945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:31.191013 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:31.191144 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:31.191144 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:31.191314 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:31.191365 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.191367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.191997 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.191998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.192595 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.192598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.192925 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.192925 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.192965 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.193483 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:31.193488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:31.195466 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
W20260811 07:07:31.195660 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:31.190872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:50569 (local address 127.3.236.190:44805)
0811 07:07:31.190957 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:31.190960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:31.191013 (+ 53us) server_negotiation.cc:407] Connection header received
0811 07:07:31.191072 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:31.191073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:31.191084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:31.191124 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:31.191432 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.191930 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.192670 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.192857 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.192916 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.195560 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:50569: BlockingRecv error: recv got EOF from 127.3.236.130:50569 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:31 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:31.391171 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:31.386404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:48807)
0811 07:07:31.386616 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:31.386627 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:31.386689 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:31.386825 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:31.386826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:31.386997 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:31.387049 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.387051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.387811 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.387812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.388405 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:31.388408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.388743 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.388744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:31.388797 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.389250 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:31.389254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:31.391092 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":142,"threads_started":1}
W20260811 07:07:31.391352 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:31.386549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:48807 (local address 127.3.236.190:44805)
0811 07:07:31.386614 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:31.386616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:31.386688 (+ 72us) server_negotiation.cc:407] Connection header received
0811 07:07:31.386734 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:31.386734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:31.386745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:31.386801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:31.387164 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.387731 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.388478 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:31.388664 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:31.388724 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:31.391212 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:48807: BlockingRecv error: recv got EOF from 127.3.236.131:48807 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:32.079450 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:32.074573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:33149)
0811 07:07:32.074716 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:32.074734 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:32.074795 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:32.074926 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:32.074927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:32.075089 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:32.075140 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.075142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.075871 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.075872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.076490 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.076492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.076888 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.076889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.076953 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.077386 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:32.077391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:32.079349 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260811 07:07:32.079509 19883 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:32.074616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:33149 (local address 127.3.236.190:44805)
0811 07:07:32.074754 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:32.074756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:32.074778 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:07:32.074838 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:32.074839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:32.074849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:32.074910 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:32.075201 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.075784 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.076594 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.076797 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.076872 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.079434 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:33149: BlockingRecv error: recv got EOF from 127.3.236.129:33149 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:32.200485 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:32.196045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:45383)
0811 07:07:32.196237 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:32.196269 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:32.196329 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:32.196455 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:32.196455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:32.196630 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:32.196679 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.196681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.197323 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.197324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.197850 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.197852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.198179 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.198180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.198219 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.198661 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:32.198665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:32.200426 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":119,"threads_started":1}
W20260811 07:07:32.200610 19883 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:32.196102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:45383 (local address 127.3.236.190:44805)
0811 07:07:32.196171 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:32.196173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:32.196330 (+ 157us) server_negotiation.cc:407] Connection header received
0811 07:07:32.196375 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:32.196376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:32.196387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:32.196435 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:32.196744 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.197246 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.197923 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.198116 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.198165 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.200527 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:45383: BlockingRecv error: recv got EOF from 127.3.236.130:45383 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:32 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:32.396145 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:32.391641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:58237)
0811 07:07:32.391771 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:32.391790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:32.391853 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:32.391982 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:32.391983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:32.392156 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:32.392208 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.392210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.392882 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.392883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.393418 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:32.393419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.393744 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.393745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:32.393791 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.394220 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:32.394225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:32.396082 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
W20260811 07:07:32.396289 19883 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:32.391722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:58237 (local address 127.3.236.190:44805)
0811 07:07:32.391773 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:32.391775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:32.391859 (+ 84us) server_negotiation.cc:407] Connection header received
0811 07:07:32.391913 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:32.391914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:32.391927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:32.391969 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:32.392270 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.392807 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.393493 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:32.393676 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:32.393727 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:32.396186 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:58237: BlockingRecv error: recv got EOF from 127.3.236.131:58237 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:33.085160 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:33.080156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:35043)
0811 07:07:33.080319 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:33.080334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:33.080412 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:33.080585 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:33.080585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:33.080750 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:33.080844 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.080848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.081673 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.081673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.082256 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.082258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.082661 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.082662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.082702 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.083191 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:33.083196 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:33.085083 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
W20260811 07:07:33.085228 19891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:33.080177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:35043 (local address 127.3.236.190:44805)
0811 07:07:33.080353 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:33.080356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:33.080387 (+ 31us) server_negotiation.cc:407] Connection header received
0811 07:07:33.080505 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:33.080505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:33.080528 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:33.080580 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:33.080935 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.081599 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.082343 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.082589 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.082655 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.085174 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:35043: BlockingRecv error: recv got EOF from 127.3.236.129:35043 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:33.209955 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:33.201070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:53609)
0811 07:07:33.201216 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:33.201228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:33.201300 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:33.201438 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:33.201438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:33.201611 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:33.201662 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.201665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.202305 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.202306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.202847 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.202849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.203229 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.203230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.203280 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.207923 (+ 4643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:33.207929 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:33.209838 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260811 07:07:33.209992 19891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:33.201121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:53609 (local address 127.3.236.190:44805)
0811 07:07:33.201191 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:33.201193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:33.201286 (+ 93us) server_negotiation.cc:407] Connection header received
0811 07:07:33.201345 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:33.201346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:33.201357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:33.201409 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:33.201730 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.202225 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.202922 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.203138 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.203196 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.209930 (+ 6734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:53609: BlockingRecv error: recv got EOF from 127.3.236.130:53609 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:33 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:33.401506 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:33.396660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:52161)
0811 07:07:33.396821 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:33.396833 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:33.396901 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:33.397034 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:33.397036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:33.397212 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:33.397262 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.397264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.398028 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.398029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.398558 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:33.398559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.399001 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.399002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:33.399056 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.399566 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:33.399571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:33.401447 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
W20260811 07:07:33.401648 19891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:33.396740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52161 (local address 127.3.236.190:44805)
0811 07:07:33.396810 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:33.396812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:33.396902 (+ 90us) server_negotiation.cc:407] Connection header received
0811 07:07:33.396944 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:33.396945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:33.396956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:33.397006 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:33.397362 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.397947 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.398657 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:33.398868 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:33.398969 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:33.401540 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:52161: BlockingRecv error: recv got EOF from 127.3.236.131:52161 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:34.090816 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:34.085685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:57519)
0811 07:07:34.085859 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:34.085873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:34.085989 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:34.086145 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:34.086146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:34.086315 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:34.086366 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.086367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.087018 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.087019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.087751 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.087769 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.088159 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.088159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.088237 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.088757 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:34.088763 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:34.090757 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
W20260811 07:07:34.090951 19899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:34.085806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:57519 (local address 127.3.236.190:44805)
0811 07:07:34.085981 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:34.085983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:34.085993 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:34.086053 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:34.086053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:34.086064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:34.086126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:34.086438 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.086926 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.087834 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.088053 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.088115 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.090854 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:57519: BlockingRecv error: recv got EOF from 127.3.236.129:57519 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:34.215528 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:34.210576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:46951)
0811 07:07:34.210765 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:34.210778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:34.210848 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:34.210985 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:34.210986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:34.211154 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:34.211206 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.211207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.211832 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.211832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.212528 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.212531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.212858 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.212859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.212898 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.213421 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:34.213427 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:34.215468 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
W20260811 07:07:34.215755 19899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:34.210708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:46951 (local address 127.3.236.190:44805)
0811 07:07:34.210772 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:34.210774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:34.210844 (+ 70us) server_negotiation.cc:407] Connection header received
0811 07:07:34.210903 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:34.210903 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:34.210914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:34.210962 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:34.211264 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.211756 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.212606 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.212792 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.212848 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.215603 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:46951: BlockingRecv error: recv got EOF from 127.3.236.130:46951 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:34 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:34.406622 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:34.401981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:35111)
0811 07:07:34.402148 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:34.402160 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:34.402231 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:34.402394 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:34.402396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:34.402638 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:34.402690 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.402692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.403321 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.403322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.403875 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:34.403877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.404192 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.404193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:34.404234 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.404792 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:34.404796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:34.406560 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
W20260811 07:07:34.406739 19899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:34.402033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:35111 (local address 127.3.236.190:44805)
0811 07:07:34.402119 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:34.402121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:34.402233 (+ 112us) server_negotiation.cc:407] Connection header received
0811 07:07:34.402300 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:34.402300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:34.402315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:34.402388 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:34.402757 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.403238 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.403950 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:34.404126 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:34.404175 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:34.406662 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:35111: BlockingRecv error: recv got EOF from 127.3.236.131:35111 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:35.096088 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:35.091418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:40061)
0811 07:07:35.091580 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:35.091590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:35.091651 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:35.091792 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:35.091792 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:35.091972 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:35.092024 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.092026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.092733 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.092734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.093265 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.093267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.093616 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.093617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.093657 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.094098 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:35.094103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:35.096023 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
W20260811 07:07:35.096189 19905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:35.091464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:40061 (local address 127.3.236.190:44805)
0811 07:07:35.091604 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:35.091606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:35.091660 (+ 54us) server_negotiation.cc:407] Connection header received
0811 07:07:35.091690 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:35.091691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:35.091701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:35.091772 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:35.092097 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.092658 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.093348 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.093550 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.093604 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.096112 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:40061: BlockingRecv error: recv got EOF from 127.3.236.129:40061 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:35.220921 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:35.216059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:47935)
0811 07:07:35.216185 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:35.216196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:35.216267 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:35.216401 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:35.216402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:35.216576 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:35.216627 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.216628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.217252 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.217253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.217795 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.217797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.218244 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.218245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.218288 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.218814 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:35.218819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:35.220819 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
W20260811 07:07:35.220966 19905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:35.216131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47935 (local address 127.3.236.190:44805)
0811 07:07:35.216204 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:35.216223 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:35.216239 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:07:35.216323 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:35.216324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:35.216334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:35.216383 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:35.216677 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.217184 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.217866 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.218179 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.218233 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.220904 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:47935: BlockingRecv error: recv got EOF from 127.3.236.130:47935 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:35 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:35.411955 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:35.407265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:49357)
0811 07:07:35.407445 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:35.407456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:35.407525 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:35.407647 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:35.407648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:35.407823 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:35.407870 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.407872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.408560 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.408561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.409108 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:35.409110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.409451 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.409452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:35.409489 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.409945 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:35.409949 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:35.411819 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
W20260811 07:07:35.411998 19905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:35.407346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:49357 (local address 127.3.236.190:44805)
0811 07:07:35.407405 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:35.407407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:35.407513 (+ 106us) server_negotiation.cc:407] Connection header received
0811 07:07:35.407582 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:35.407582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:35.407594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:35.407643 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:35.407969 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.408478 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.409190 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:35.409383 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:35.409444 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:35.411920 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:49357: BlockingRecv error: recv got EOF from 127.3.236.131:49357 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:36.101620 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:36.096698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:35587)
0811 07:07:36.096925 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:36.096940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:36.097011 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:36.097157 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:36.097158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:36.097338 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:36.097391 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.097393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.098035 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.098036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.098566 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.098568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.098952 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.098953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.099004 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.099464 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:36.099468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:36.101544 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
W20260811 07:07:36.101691 19913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:36.096812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:35587 (local address 127.3.236.190:44805)
0811 07:07:36.096975 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:36.096977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:36.096997 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:07:36.097072 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:36.097072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:36.097083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:36.097133 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:36.097457 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.097954 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.098640 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.098832 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.098882 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.101638 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:35587: BlockingRecv error: recv got EOF from 127.3.236.129:35587 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:36.226543 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:36.221433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:42265)
0811 07:07:36.221635 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:36.221649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:36.221749 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:36.221947 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:36.221948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:36.222179 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:36.222241 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.222244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.222989 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.222990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.223604 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.223607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.224030 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.224032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.224098 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.224614 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:36.224619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:36.226484 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
W20260811 07:07:36.226845 19913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:36.221526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:42265 (local address 127.3.236.190:44805)
0811 07:07:36.221616 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:36.221619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:36.221729 (+ 110us) server_negotiation.cc:407] Connection header received
0811 07:07:36.221828 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:36.221829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:36.221841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:36.221926 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:36.222336 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.222885 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.223706 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.223929 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.224017 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.226793 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:42265: BlockingRecv error: recv got EOF from 127.3.236.130:42265 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:36 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:36.417768 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:36.412552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:52675)
0811 07:07:36.412730 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:36.412741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:36.412807 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:36.412967 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:36.412968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:36.413169 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:36.413222 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.413223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.414061 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.414062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.414687 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:36.414689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.415026 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.415027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:36.415068 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.415622 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:36.415627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:36.417641 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
W20260811 07:07:36.417832 19913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:36.412617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52675 (local address 127.3.236.190:44805)
0811 07:07:36.412686 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:36.412689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:36.412808 (+ 119us) server_negotiation.cc:407] Connection header received
0811 07:07:36.412852 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:36.412852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:36.412865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:36.412918 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:36.413299 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.413946 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.414761 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:36.414959 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:36.415012 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:36.417772 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:52675: BlockingRecv error: recv got EOF from 127.3.236.131:52675 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:37.106894 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:37.102139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:51757)
0811 07:07:37.102272 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:37.102308 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:37.102358 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:37.102589 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:37.102590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:37.102826 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:37.102877 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.102879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.103583 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.103583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.104103 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.104104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.104513 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.104513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.104560 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.105000 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:37.105005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:37.106830 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260811 07:07:37.107038 19920 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:37.102286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:51757 (local address 127.3.236.190:44805)
0811 07:07:37.102446 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:37.102448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:37.102456 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:37.102478 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:37.102479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:37.102505 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:37.102569 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:37.102982 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.103505 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.104189 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.104446 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.104510 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.106944 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:51757: BlockingRecv error: recv got EOF from 127.3.236.129:51757 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:37.233121 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:37.227114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:58607)
0811 07:07:37.227318 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:37.227330 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:37.227407 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:37.227591 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:37.227592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:37.227788 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:37.227838 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.227840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.228770 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.228771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.229346 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.229348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.229724 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.229725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.229786 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.230351 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:37.230356 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:37.232990 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":106,"threads_started":1}
W20260811 07:07:37.233217 19920 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:37.227225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:58607 (local address 127.3.236.190:44805)
0811 07:07:37.227315 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:37.227318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:37.227384 (+ 66us) server_negotiation.cc:407] Connection header received
0811 07:07:37.227473 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:37.227474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:37.227505 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:37.227583 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:37.227929 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.228684 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.229445 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.229647 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.229723 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.233138 (+ 3415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:58607: BlockingRecv error: recv got EOF from 127.3.236.130:58607 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:37 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:37.423336 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:37.418370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:42217)
0811 07:07:37.418516 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:37.418527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:37.418592 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:37.418721 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:37.418723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:37.418977 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:37.419040 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.419043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.419825 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.419826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.420445 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:37.420447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.420799 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.420800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:37.420851 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.421371 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:37.421375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:37.423259 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260811 07:07:37.423405 19920 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:37.418437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:42217 (local address 127.3.236.190:44805)
0811 07:07:37.418522 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:37.418525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:37.418577 (+ 52us) server_negotiation.cc:407] Connection header received
0811 07:07:37.418636 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:37.418637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:37.418648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:37.418704 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:37.419126 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.419742 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.420520 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:37.420703 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:37.420761 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:37.423355 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:42217: BlockingRecv error: recv got EOF from 127.3.236.131:42217 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:38.112421 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:38.107482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:40913)
0811 07:07:38.107643 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:38.107656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:38.107741 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:38.107960 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:38.107961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:38.108237 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:38.108330 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.108332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.109019 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.109020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.109544 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.109545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.109891 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.109892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.109933 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.110388 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:38.110393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:38.112320 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
W20260811 07:07:38.112466 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:38.107579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:40913 (local address 127.3.236.190:44805)
0811 07:07:38.107736 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:38.107768 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:38.107798 (+ 30us) server_negotiation.cc:407] Connection header received
0811 07:07:38.107827 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:38.107828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:38.107843 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:38.107923 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:38.108424 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.108944 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.109627 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.109824 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.109879 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.112395 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:40913: BlockingRecv error: recv got EOF from 127.3.236.129:40913 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:38.244580 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:38.235221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:34257)
0811 07:07:38.235371 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:38.235400 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:38.235471 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:38.235639 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:38.235640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:38.235802 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:38.235853 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.235855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.236573 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.236575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.237095 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.237097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.237468 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.237468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.237507 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.242561 (+ 5054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:38.242567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:38.244502 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
W20260811 07:07:38.244683 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:38.235313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:34257 (local address 127.3.236.190:44805)
0811 07:07:38.235424 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:38.235426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:38.235453 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:07:38.235541 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:38.235541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:38.235552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:38.235629 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:38.235925 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.236487 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.237181 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.237380 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.237456 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.244614 (+ 7158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:34257: BlockingRecv error: recv got EOF from 127.3.236.130:34257 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:38 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:38.428392 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:38.423877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:59579)
0811 07:07:38.424077 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:38.424088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:38.424154 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:38.424330 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:38.424331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:38.424513 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:38.424565 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.424567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.425209 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.425210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.425737 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:38.425739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.426073 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.426074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:38.426117 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.426562 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:38.426567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:38.428331 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":125,"threads_started":1}
W20260811 07:07:38.428493 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:38.423941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:59579 (local address 127.3.236.190:44805)
0811 07:07:38.424016 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:38.424018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:38.424154 (+ 136us) server_negotiation.cc:407] Connection header received
0811 07:07:38.424201 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:38.424201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:38.424212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:38.424317 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:38.424635 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.425136 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.425816 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:38.425996 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:38.426064 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:38.428424 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:59579: BlockingRecv error: recv got EOF from 127.3.236.131:59579 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:39.117774 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:39.112989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:49867)
0811 07:07:39.113172 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:39.113183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:39.113251 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:39.113407 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:39.113407 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:39.113602 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:39.113654 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.113656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.114301 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.114302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.114836 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.114837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.115187 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.115188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.115234 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.115690 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:39.115695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:39.117714 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":114,"threads_started":1}
W20260811 07:07:39.117913 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:39.113089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:49867 (local address 127.3.236.190:44805)
0811 07:07:39.113220 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:39.113222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:39.113231 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:39.113299 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:39.113300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:39.113310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:39.113388 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:39.113722 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.114232 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.114908 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.115122 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.115171 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.117800 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:49867: BlockingRecv error: recv got EOF from 127.3.236.129:49867 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:39.249989 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:39.245119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:45171)
0811 07:07:39.245275 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:39.245286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:39.245350 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:39.245496 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:39.245497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:39.245668 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:39.245718 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.245720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.246581 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.246582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.247116 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.247117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.247514 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.247515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.247558 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.247982 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:39.247986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:39.249920 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
W20260811 07:07:39.250156 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:39.245186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:45171 (local address 127.3.236.190:44805)
0811 07:07:39.245278 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:39.245280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:39.245335 (+ 55us) server_negotiation.cc:407] Connection header received
0811 07:07:39.245424 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:39.245424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:39.245436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:39.245478 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:39.245793 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.246474 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.247194 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.247450 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.247509 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.250076 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:45171: BlockingRecv error: recv got EOF from 127.3.236.130:45171 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:39 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:39.433904 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:39.428944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:58501)
0811 07:07:39.429095 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:39.429111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:39.429196 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:39.429359 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:39.429360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:39.429595 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:39.429646 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.429648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.430358 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.430360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.430936 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:39.430938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.431311 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.431313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:39.431383 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.431927 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:39.431932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:39.433825 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260811 07:07:39.434043 19929 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:39.429106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:58501 (local address 127.3.236.190:44805)
0811 07:07:39.429189 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:39.429191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:39.429202 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:07:39.429264 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:39.429269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:39.429282 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:39.429337 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:39.429735 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.430264 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.431036 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:39.431233 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:39.431311 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:39.433918 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:58501: BlockingRecv error: recv got EOF from 127.3.236.131:58501 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:40.123090 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:40.118322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:44971)
0811 07:07:40.118536 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:40.118564 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:40.118637 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:40.118785 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:40.118786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:40.118985 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:40.119033 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.119035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.119797 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.119798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.120415 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.120417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.120762 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.120763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.120802 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.121241 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:40.121246 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:40.123005 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
W20260811 07:07:40.123214 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:40.118362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:44971 (local address 127.3.236.190:44805)
0811 07:07:40.118548 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:40.118551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:40.118624 (+ 73us) server_negotiation.cc:407] Connection header received
0811 07:07:40.118696 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:40.118697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:40.118708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:40.118765 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:40.119129 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.119726 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.120495 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.120697 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.120750 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.123130 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:44971: BlockingRecv error: recv got EOF from 127.3.236.129:44971 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:40.255131 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:40.250480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:40339)
0811 07:07:40.250670 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:40.250680 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:40.250741 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:40.250883 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:40.250884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:40.251078 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:40.251126 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.251127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.251768 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.251769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.252394 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.252396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.252724 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.252725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.252763 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.253224 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:40.253229 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:40.255058 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
W20260811 07:07:40.255266 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:40.250551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:40339 (local address 127.3.236.190:44805)
0811 07:07:40.250675 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:40.250678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:40.250747 (+ 69us) server_negotiation.cc:407] Connection header received
0811 07:07:40.250791 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:40.250792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:40.250804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:40.250863 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:40.251199 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.251697 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.252473 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.252658 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.252713 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.255155 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:40339: BlockingRecv error: recv got EOF from 127.3.236.130:40339 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:40 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:40.438899 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:40.434402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:45983)
0811 07:07:40.434583 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:40.434595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:40.434656 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:40.434791 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:40.434792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:40.434968 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:40.435017 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.435019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.435657 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.435658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.436198 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:40.436200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.436534 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.436535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:40.436592 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.437074 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:40.437078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:40.438827 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":64,"threads_started":1}
W20260811 07:07:40.439034 19933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:40.434574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:45983 (local address 127.3.236.190:44805)
0811 07:07:40.434652 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:40.434654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:40.434691 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:07:40.434713 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:40.434713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:40.434726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:40.434765 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:40.435088 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.435589 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.436285 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:40.436468 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:40.436522 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:40.438929 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:45983: BlockingRecv error: recv got EOF from 127.3.236.131:45983 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:41.128737 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:41.123609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:33619)
0811 07:07:41.123774 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:41.123784 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:41.123837 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:41.124049 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:41.124051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:41.124242 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:41.124369 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.124371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.125071 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.125072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.125598 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.125599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.126123 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.126125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.126197 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.126802 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:41.126808 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:41.128677 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260811 07:07:41.128892 19942 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:41.123695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:33619 (local address 127.3.236.190:44805)
0811 07:07:41.123828 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:41.123830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:41.123838 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:41.123900 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:41.123901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:41.123912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:41.123968 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:41.124468 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.124975 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.125710 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.125991 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.126057 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.128795 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:33619: BlockingRecv error: recv got EOF from 127.3.236.129:33619 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:41.260172 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:41.255673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:41005)
0811 07:07:41.255823 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:41.255835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:41.255911 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:41.256096 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:41.256097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:41.256340 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:41.256390 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.256392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.257018 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.257019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.257553 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.257554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.257879 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.257879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.257919 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.258352 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:41.258357 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:41.260114 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260811 07:07:41.260340 19942 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:41.255730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:41005 (local address 127.3.236.190:44805)
0811 07:07:41.255836 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:41.255838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:41.255905 (+ 67us) server_negotiation.cc:407] Connection header received
0811 07:07:41.255981 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:41.255982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:41.255997 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:41.256078 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:41.256458 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.256946 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.257628 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.257813 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.257865 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.260235 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:41005: BlockingRecv error: recv got EOF from 127.3.236.130:41005 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:41 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:41.444608 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:41.439468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:57157)
0811 07:07:41.439639 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:41.439650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:41.439713 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:41.439857 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:41.439858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:41.440046 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:41.440093 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.440095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.441136 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.441138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.441755 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:41.441758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.442107 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.442107 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:41.442153 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.442603 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:41.442609 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:41.444546 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
W20260811 07:07:41.444749 19942 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:41.439590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:57157 (local address 127.3.236.190:44805)
0811 07:07:41.439647 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:41.439649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:41.439698 (+ 49us) server_negotiation.cc:407] Connection header received
0811 07:07:41.439761 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:41.439761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:41.439773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:41.439834 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:41.440182 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.441005 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.441831 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:41.442025 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:41.442086 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:41.444664 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:57157: BlockingRecv error: recv got EOF from 127.3.236.131:57157 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:42.133865 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:42.129228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:40125)
0811 07:07:42.129376 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:42.129396 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:42.129448 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:42.129587 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:42.129588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:42.129762 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:42.129829 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.129831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.130523 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.130523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.131049 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.131050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.131418 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.131419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.131462 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.131914 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:42.131918 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:42.133805 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
W20260811 07:07:42.134011 19949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:42.129291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:40125 (local address 127.3.236.190:44805)
0811 07:07:42.129427 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:42.129429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:42.129438 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:42.129496 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:42.129496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:42.129506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:42.129574 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:42.129916 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.130442 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.131135 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.131344 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.131402 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.133916 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:40125: BlockingRecv error: recv got EOF from 127.3.236.129:40125 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:42.265928 19950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:42.260705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:47097)
0811 07:07:42.260859 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:42.260889 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:42.260956 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:42.261169 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:42.261170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:42.261365 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:42.261412 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.261414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.262096 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.262097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.262658 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.262660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.263044 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.263045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.263099 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.263606 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:42.263611 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:42.265805 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260811 07:07:42.265955 19949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:42.260850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47097 (local address 127.3.236.190:44805)
0811 07:07:42.260926 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:42.260928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:42.260938 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:42.261067 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:42.261068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:42.261082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:42.261158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:42.261497 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.261998 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.262749 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.262966 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.263052 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.265886 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:47097: BlockingRecv error: recv got EOF from 127.3.236.130:47097 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:42 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:42.449944 19951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:42.445115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:54129)
0811 07:07:42.445277 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:42.445290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:42.445357 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:42.445579 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:42.445580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:42.445774 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:42.445825 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.445827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.446480 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.446480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.447119 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:42.447121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.447443 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.447444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:42.447489 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.447947 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:42.447951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:42.449873 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
W20260811 07:07:42.450016 19949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:42.445218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:54129 (local address 127.3.236.190:44805)
0811 07:07:42.445302 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:42.445304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:42.445342 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:07:42.445419 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:42.445420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:42.445433 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:42.445520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:42.445909 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.446401 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.447197 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:42.447378 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:42.447430 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:42.449967 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:54129: BlockingRecv error: recv got EOF from 127.3.236.131:54129 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:43.139108 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:43.134432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:43213)
0811 07:07:43.134593 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:43.134605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:43.134661 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:43.134825 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:43.134826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:43.135027 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:43.135076 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.135078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.135719 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.135720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.136307 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.136309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.136644 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.136645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.136689 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.137215 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:43.137223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:43.139043 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260811 07:07:43.139215 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:43.134521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:43213 (local address 127.3.236.190:44805)
0811 07:07:43.134634 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:43.134636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:43.134646 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:43.134733 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:43.134734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:43.134745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:43.134803 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:43.135147 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.135652 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.136376 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.136574 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.136626 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.139138 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:43213: BlockingRecv error: recv got EOF from 127.3.236.129:43213 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:43.275743 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:43.266529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:47313)
0811 07:07:43.266692 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:43.266704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:43.266770 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:43.266904 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:43.266905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:43.267083 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:43.267131 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.267132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.267836 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.267837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.268413 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.268415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.268785 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.268786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.268826 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.273814 (+ 4988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:43.273819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:43.275680 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
W20260811 07:07:43.275859 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:43.266598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47313 (local address 127.3.236.190:44805)
0811 07:07:43.266668 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:43.266671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:43.266768 (+ 97us) server_negotiation.cc:407] Connection header received
0811 07:07:43.266829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:43.266830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:43.266842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:43.266884 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:43.267205 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.267754 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.268516 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.268714 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.268777 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.275784 (+ 7007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:47313: BlockingRecv error: recv got EOF from 127.3.236.130:47313 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:43 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.131@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:43.455029 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:43.450493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.131:50803)
0811 07:07:43.450652 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:43.450663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:43.450725 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:43.450858 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:43.450859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:43.451036 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:43.451084 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.451086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.451720 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.451720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.452321 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:43.452323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.452666 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.452666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:43.452706 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.453162 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:43.453166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:43.454967 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
W20260811 07:07:43.455130 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:43.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:50803 (local address 127.3.236.190:44805)
0811 07:07:43.450623 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:43.450625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:43.450726 (+ 101us) server_negotiation.cc:407] Connection header received
0811 07:07:43.450780 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:43.450781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:43.450796 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:43.450837 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:43.451152 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.451651 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.452398 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:43.452599 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:43.452655 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:43.455061 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.131:50803: BlockingRecv error: recv got EOF from 127.3.236.131:50803 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 07:07:44 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:44 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.129@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:44.144582 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.139729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.129:36419)
0811 07:07:44.139870 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:44.139883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:44.139952 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:44.140103 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:44.140104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:44.140320 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:44.140375 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.140377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.141068 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.141070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.141615 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.141617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.141981 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.141982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.142024 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.142652 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:44.142657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:44.144520 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260811 07:07:44.144733 19964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:44.139842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:36419 (local address 127.3.236.190:44805)
0811 07:07:44.139971 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:44.139973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:44.139982 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:44.140004 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:44.140004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:44.140014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:44.140075 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:44.140456 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.140977 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.141694 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.141910 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.141963 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.144642 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.129:36419: BlockingRecv error: recv got EOF from 127.3.236.129:36419 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
Aug 11 07:07:44 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:07:44 dist-test-slave-2d19 krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.236.130@KRBTEST.COM for oryx/127.3.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:07:44.281070 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.276373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:44805 (local address 127.3.236.130:40715)
0811 07:07:44.276579 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:44.276598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:44.276668 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:44.276801 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:44.276802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:44.276973 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:44.277029 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.277031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.277658 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.277658 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.278195 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.278196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.278536 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.278537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.278577 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.279014 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:44.279020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:44.280959 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.236.190:44805: Server oryx/127.3.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
W20260811 07:07:44.281205 19964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:44.276441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:40715 (local address 127.3.236.190:44805)
0811 07:07:44.276558 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:44.276560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:44.276651 (+ 91us) server_negotiation.cc:407] Connection header received
0811 07:07:44.276717 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:44.276718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:44.276728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:44.276778 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:44.277092 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.277589 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.278267 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.278453 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.278519 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.281106 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.236.130:40715: BlockingRecv error: recv got EOF from 127.3.236.130:40715 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:07:44.356201 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 19123
I20260811 07:07:44.391119 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 19371
I20260811 07:07:44.408170 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 19609
I20260811 07:07:44.423908 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 19015
2026-08-11T07:07:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17889 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T07:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:44Z Disabled control of system clock
I20260811 07:07:44.477214 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35249
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 07:07:44.578615 19971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:44.578833 19971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:44.578855 19971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:44.580001 19971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:44.580041 19971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:44.580067 19971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:44.580080 19971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:44.580093 19971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:44.581588 19971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:44.581882 19971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:44.582217 19971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:44.588841 19978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:44.588938 19971 server_base.cc:1034] running on GCE node
W20260811 07:07:44.588852 19977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:44.590806 19981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:44.591109 19971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:44.591373 19971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:44.592519 19971 hybrid_clock.cc:648] HybridClock initialized: now 1786432064592496 us; error 43 us; skew 500 ppm
I20260811 07:07:44.594070 19971 webserver.cc:460] Webserver started at http://127.3.236.190:46641/ using document root <none> and password file <none>
I20260811 07:07:44.594385 19971 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:44.594492 19971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:44.594652 19971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:44.595849 19971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "fe41a5b02442434e97a532856817b7e5"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ab8089d06764c69c2c2c43a50defc19d"
server_key_iv: "ea09677962ab8be0dd024bb979c31454"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.596372 19971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "fe41a5b02442434e97a532856817b7e5"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ab8089d06764c69c2c2c43a50defc19d"
server_key_iv: "ea09677962ab8be0dd024bb979c31454"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.597889 19971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:44.598858 19993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.599086 19971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:44.599195 19971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "fe41a5b02442434e97a532856817b7e5"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ab8089d06764c69c2c2c43a50defc19d"
server_key_iv: "ea09677962ab8be0dd024bb979c31454"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.599287 19971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:44.619263 19971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:44.619663 19971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:44.619832 19971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:44.624477 19971 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35249
I20260811 07:07:44.624511 20085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35249 every 8 connection(s)
I20260811 07:07:44.624876 19971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:44.625741 20086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:44.628696 20086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5: Bootstrap starting.
I20260811 07:07:44.629638 20086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:44.630069 20086 log.cc:824] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:44.630831 20086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5: No bootstrap required, opened a new log
I20260811 07:07:44.632434 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 19971
I20260811 07:07:44.632543 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:44.632735 4018 external_mini_cluster.cc:1428] Setting key 81aaa3fa4d4eecb60606698f27c5ebb7
I20260811 07:07:44.632855 20086 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } }
I20260811 07:07:44.633095 20086 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:44.633188 20086 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe41a5b02442434e97a532856817b7e5, State: Initialized, Role: FOLLOWER
I20260811 07:07:44.633329 20086 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } }
I20260811 07:07:44.633659 20086 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:44.633864 20086 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:44.633962 20086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:44.634789 20086 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } }
I20260811 07:07:44.634979 20086 leader_election.cc:302] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe41a5b02442434e97a532856817b7e5; no voters:
I20260811 07:07:44.635154 20086 leader_election.cc:288] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:44.635272 20093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:44.635496 20093 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [term 1 LEADER]: Becoming Leader. State: Replica: fe41a5b02442434e97a532856817b7e5, State: Running, Role: LEADER
I20260811 07:07:44.635521 20086 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:44.635653 20093 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } }
I20260811 07:07:44.636169 20094 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe41a5b02442434e97a532856817b7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } } }
I20260811 07:07:44.636250 20094 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:44.636509 20095 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe41a5b02442434e97a532856817b7e5. Latest consensus state: current_term: 1 leader_uuid: "fe41a5b02442434e97a532856817b7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe41a5b02442434e97a532856817b7e5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35249 } } }
I20260811 07:07:44.636868 20095 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:44.637030 20106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:44.637835 20106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:44.639647 20106 catalog_manager.cc:1347] Generated new cluster ID: accd6aef5e7b4a348912a88de7fc3ccf
I20260811 07:07:44.639710 20106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:44.640308 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.633306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.3.236.190:35249)
0811 07:07:44.633585 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:44.633605 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:44.633642 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:07:44.633765 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:44.633773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:44.633847 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:44.634007 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:44.634263 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.636329 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.637636 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.638421 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.638613 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.639359 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:44.639363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:44.639364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:44.639597 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:44.639644 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:44.639661 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:44.639726 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:44.639889 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:07:44.639985 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
I20260811 07:07:44.655642 20106 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:44.656235 20106 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:44.665911 20106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe41a5b02442434e97a532856817b7e5: Generated new TSK 0
I20260811 07:07:44.666127 20106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:44.669642 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--builtin_ntp_servers=127.3.236.148:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 07:07:44.757458 20120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:44.757686 20120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:44.757709 20120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:44.758898 20120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:44.758945 20120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:44.758970 20120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:44.760427 20120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:44.760682 20120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:44.761018 20120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:44.767244 20135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:44.767234 20137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:44.767302 20129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:44.767721 20120 server_base.cc:1034] running on GCE node
I20260811 07:07:44.767989 20120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:44.768301 20120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:44.769518 20120 hybrid_clock.cc:648] HybridClock initialized: now 1786432064769497 us; error 34 us; skew 500 ppm
I20260811 07:07:44.771117 20120 webserver.cc:460] Webserver started at http://127.3.236.129:39937/ using document root <none> and password file <none>
I20260811 07:07:44.771395 20120 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:44.771445 20120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:44.771585 20120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:44.772833 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "383f2902881240ad972836b9cb836c35"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ffaf62a3056e80a752b3181c56754fb0"
server_key_iv: "b3a0b466a0ad51de72440b70922bd363"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.773231 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "383f2902881240ad972836b9cb836c35"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ffaf62a3056e80a752b3181c56754fb0"
server_key_iv: "b3a0b466a0ad51de72440b70922bd363"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.774682 20120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:44.775537 20147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.775697 20120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:44.775810 20120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "383f2902881240ad972836b9cb836c35"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "ffaf62a3056e80a752b3181c56754fb0"
server_key_iv: "b3a0b466a0ad51de72440b70922bd363"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.775885 20120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:44.802801 20120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:44.803170 20120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:44.803329 20120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:44.803583 20120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:44.804114 20120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:44.804152 20120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.804176 20120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:44.804188 20120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.810542 20120 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:41645
I20260811 07:07:44.810635 20336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:41645 every 8 connection(s)
I20260811 07:07:44.810969 20120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:44.815845 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 20120
I20260811 07:07:44.815953 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:44.816185 4018 external_mini_cluster.cc:1428] Setting key d58548892f44aa8d789932367c5f659a
I20260811 07:07:44.816350 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.811858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:50773 (local address 127.3.236.190:35249)
0811 07:07:44.811916 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:44.811918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:44.812198 (+ 280us) server_negotiation.cc:407] Connection header received
0811 07:07:44.812703 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:44.812704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:44.812717 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:44.812767 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:44.813460 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.813949 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.814905 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.815094 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.815147 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.815991 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:44.815993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:44.815994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:44.816002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:44.816019 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:44.816024 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:44.816103 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:44.816213 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 07:07:44.816247 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 07:07:44.816598 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.811847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35249 (local address 127.3.236.129:50773)
0811 07:07:44.812073 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:44.812107 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:44.812611 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:44.812830 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:44.812833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:44.812841 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:44.813364 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.813369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.814056 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.814056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.814826 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.814828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.815172 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.815173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.815285 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.815814 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:44.815814 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:44.815856 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:44.815885 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:44.816108 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:44.816117 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:44.816121 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:44.816213 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:07:44.816334 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260811 07:07:44.817132 20337 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35249
I20260811 07:07:44.817243 20337 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:44.817432 20337 heartbeater.cc:511] Master 127.3.236.190:35249 requested a full tablet report, sending...
I20260811 07:07:44.817535 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--builtin_ntp_servers=127.3.236.148:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 07:07:44.818166 20018 ts_manager.cc:194] Registered new tserver with Master: 383f2902881240ad972836b9cb836c35 (127.3.236.129:41645)
I20260811 07:07:44.818830 20018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.129:50773
W20260811 07:07:44.903270 20345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:44.903481 20345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:44.903504 20345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:44.904625 20345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:44.904664 20345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:44.904690 20345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:44.905959 20345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:44.906173 20345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:44.906435 20345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:44.911585 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:44.911741 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:44.911835 20345 server_base.cc:1034] running on GCE node
W20260811 07:07:44.911607 20356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:44.912210 20345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:44.912479 20345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:44.913657 20345 hybrid_clock.cc:648] HybridClock initialized: now 1786432064913644 us; error 27 us; skew 500 ppm
I20260811 07:07:44.915005 20345 webserver.cc:460] Webserver started at http://127.3.236.130:37795/ using document root <none> and password file <none>
I20260811 07:07:44.915313 20345 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:44.915390 20345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:44.915549 20345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:44.916800 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "a9744e2e7eb94736ac3abefb73e78c4d"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "f25e26842a2e9d3470817c9bf17c38b4"
server_key_iv: "e961aec49a740d9100136f0005f674e7"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.917182 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "a9744e2e7eb94736ac3abefb73e78c4d"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "f25e26842a2e9d3470817c9bf17c38b4"
server_key_iv: "e961aec49a740d9100136f0005f674e7"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.918601 20345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:07:44.919334 20366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.919538 20345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:44.919664 20345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "a9744e2e7eb94736ac3abefb73e78c4d"
format_stamp: "Formatted at 2026-08-11 07:07:44 on dist-test-slave-2d19"
server_key: "f25e26842a2e9d3470817c9bf17c38b4"
server_key_iv: "e961aec49a740d9100136f0005f674e7"
server_key_version: "encryptionkey@0"
I20260811 07:07:44.919749 20345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:44.946085 20345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:44.946456 20345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:44.946624 20345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:44.946902 20345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:44.947284 20345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:44.947341 20345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.947379 20345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:44.947433 20345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:44.954047 20345 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41327
I20260811 07:07:44.954098 20559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41327 every 8 connection(s)
I20260811 07:07:44.954461 20345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:44.959662 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.955229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:60673 (local address 127.3.236.190:35249)
0811 07:07:44.955297 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:44.955298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:44.955539 (+ 241us) server_negotiation.cc:407] Connection header received
0811 07:07:44.956058 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:44.956060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:44.956069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:44.956132 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:44.956828 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.957324 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.958328 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.958505 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.958555 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.959365 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:44.959366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:44.959366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:44.959371 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:44.959384 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:44.959388 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:44.959431 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:44.959576 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:07:44.959601 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:07:44.959966 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:44.955211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35249 (local address 127.3.236.130:60673)
0811 07:07:44.955417 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:44.955461 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:44.955959 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:44.956187 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:44.956191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:44.956199 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:44.956749 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.956755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.957433 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.957434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.958245 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:44.958248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:44.958580 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:44.958581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:44.958699 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:44.959215 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:44.959215 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:44.959245 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:44.959269 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:44.959481 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:44.959488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:44.959491 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:44.959566 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:07:44.959649 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260811 07:07:44.960559 20560 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35249
I20260811 07:07:44.960655 20560 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:44.960865 20560 heartbeater.cc:511] Master 127.3.236.190:35249 requested a full tablet report, sending...
I20260811 07:07:44.961275 20018 ts_manager.cc:194] Registered new tserver with Master: a9744e2e7eb94736ac3abefb73e78c4d (127.3.236.130:41327)
I20260811 07:07:44.961706 20018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.130:60673
I20260811 07:07:44.963820 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 20345
I20260811 07:07:44.963899 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:44.964119 4018 external_mini_cluster.cc:1428] Setting key d8740cae0004b71e5aab56b1db56129e
I20260811 07:07:44.965173 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--builtin_ntp_servers=127.3.236.148:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 07:07:45.051011 20564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:45.051275 20564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:45.051314 20564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:45.052513 20564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:45.052578 20564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:45.052620 20564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:45.053961 20564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:45.054241 20564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:45.054620 20564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:45.060532 20575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.060523 20576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.060667 20564 server_base.cc:1034] running on GCE node
W20260811 07:07:45.060523 20582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.061092 20564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:45.061367 20564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:45.062561 20564 hybrid_clock.cc:648] HybridClock initialized: now 1786432065062540 us; error 50 us; skew 500 ppm
I20260811 07:07:45.063880 20564 webserver.cc:460] Webserver started at http://127.3.236.131:36503/ using document root <none> and password file <none>
I20260811 07:07:45.064235 20564 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:45.064332 20564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:45.064479 20564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:45.065590 20564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "daaf649ee54446fe8e221d3447e47306"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "b3fc49f88e757dbcda36099970917773"
server_key_iv: "1e59d64b668d60a852a822577dbddf16"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.066042 20564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "daaf649ee54446fe8e221d3447e47306"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "b3fc49f88e757dbcda36099970917773"
server_key_iv: "1e59d64b668d60a852a822577dbddf16"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.067593 20564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:07:45.068466 20592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.068643 20564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:45.068754 20564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "daaf649ee54446fe8e221d3447e47306"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "b3fc49f88e757dbcda36099970917773"
server_key_iv: "1e59d64b668d60a852a822577dbddf16"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.068832 20564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:45.085407 20564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:45.085809 20564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:45.085989 20564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:45.086264 20564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:45.086633 20564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:45.086694 20564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.086738 20564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:45.086768 20564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.094103 20564 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:45423
I20260811 07:07:45.094244 20818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:45423 every 8 connection(s)
I20260811 07:07:45.094560 20564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:45.099783 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.095281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:44107 (local address 127.3.236.190:35249)
0811 07:07:45.095364 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.095366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.095601 (+ 235us) server_negotiation.cc:407] Connection header received
0811 07:07:45.096185 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.096186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.096198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.096323 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.097055 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:45.097568 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:45.098511 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:45.098696 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:45.098870 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:45.099538 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:45.099539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:45.099539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:45.099544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:45.099559 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:45.099563 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:45.099607 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:45.099692 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 07:07:45.099720 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:07:45.099984 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.095242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35249 (local address 127.3.236.131:44107)
0811 07:07:45.095479 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:45.095515 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:45.096068 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:45.096394 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:45.096420 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:45.096429 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:45.096944 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:45.096949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:45.097652 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:45.097653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:45.098435 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:45.098436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:45.098790 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:45.098791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:45.098913 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:45.099407 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:45.099408 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:45.099436 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:45.099459 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:45.099622 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:45.099626 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:45.099629 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:45.099682 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 07:07:45.099765 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260811 07:07:45.100536 20819 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35249
I20260811 07:07:45.100649 20819 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:45.100884 20819 heartbeater.cc:511] Master 127.3.236.190:35249 requested a full tablet report, sending...
I20260811 07:07:45.101356 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 20564
I20260811 07:07:45.101395 20018 ts_manager.cc:194] Registered new tserver with Master: daaf649ee54446fe8e221d3447e47306 (127.3.236.131:45423)
I20260811 07:07:45.101491 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:45.101723 4018 external_mini_cluster.cc:1428] Setting key 99d663d2a45f5796f01c23b35abb5d59
I20260811 07:07:45.101991 20018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.131:44107
I20260811 07:07:45.102746 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:07:45.104842 20092 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:45.104330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.3.236.190:35249)
0811 07:07:45.104394 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.104396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.104536 (+ 140us) server_negotiation.cc:407] Connection header received
0811 07:07:45.104595 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.104596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.104606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.104651 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.104780 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40938: BlockingRecv error: recv got EOF from 127.0.0.1:40938 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:07:45.105027 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 20120
I20260811 07:07:45.125568 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 20345
I20260811 07:07:45.136121 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 20564
I20260811 07:07:45.146878 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 19971
2026-08-11T07:07:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (716 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T07:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:45Z Disabled control of system clock
I20260811 07:07:45.189036 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35503
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:36425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:07:45.276630 20837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:45.276829 20837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:45.276850 20837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:45.277971 20837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:45.278013 20837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:45.278029 20837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:45.278079 20837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:45.278105 20837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:45.279449 20837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:45.279702 20837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:45.279994 20837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:45.285205 20851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.285259 20848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.285202 20847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.285375 20837 server_base.cc:1034] running on GCE node
I20260811 07:07:45.285759 20837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:45.286096 20837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:45.287256 20837 hybrid_clock.cc:648] HybridClock initialized: now 1786432065287235 us; error 40 us; skew 500 ppm
I20260811 07:07:45.288573 20837 webserver.cc:460] Webserver started at http://127.3.236.190:33865/ using document root <none> and password file <none>
I20260811 07:07:45.288808 20837 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:45.288856 20837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:45.289009 20837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:45.290182 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "8e74da8cbde244e198b16aaff4a662fe"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "07ec9ed84dc0a411437e2f23edfd986f"
server_key_iv: "8f5ca7f0b5acf55fc237d95d0dc2676d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.290591 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "8e74da8cbde244e198b16aaff4a662fe"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "07ec9ed84dc0a411437e2f23edfd986f"
server_key_iv: "8f5ca7f0b5acf55fc237d95d0dc2676d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.291890 20837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:45.292744 20870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.292923 20837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.292997 20837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "8e74da8cbde244e198b16aaff4a662fe"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "07ec9ed84dc0a411437e2f23edfd986f"
server_key_iv: "8f5ca7f0b5acf55fc237d95d0dc2676d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.293092 20837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:45.296394 20837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:45.296671 20837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:45.296808 20837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:45.300838 20837 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35503
I20260811 07:07:45.300869 20956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35503 every 8 connection(s)
I20260811 07:07:45.301194 20837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:45.301884 20958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:45.303740 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 20837
I20260811 07:07:45.303867 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:45.304061 4018 external_mini_cluster.cc:1428] Setting key 2dc6b4f267ea8e3b69540509c7d7b245
I20260811 07:07:45.305248 20958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe: Bootstrap starting.
I20260811 07:07:45.306007 20958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:45.306356 20958 log.cc:824] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:45.306828 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.304631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.3.236.190:35503)
0811 07:07:45.304858 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.304867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.304912 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:07:45.305445 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.305454 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.305505 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.305675 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.306325 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:45.306328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:45.306329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:45.306332 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:45.306372 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:45.306383 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:45.306441 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:45.306634 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:07:45.306692 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260811 07:07:45.307183 20958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe: No bootstrap required, opened a new log
I20260811 07:07:45.309098 20958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } }
I20260811 07:07:45.309259 20958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:45.309284 20958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e74da8cbde244e198b16aaff4a662fe, State: Initialized, Role: FOLLOWER
I20260811 07:07:45.309410 20958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } }
I20260811 07:07:45.309505 20958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:45.309530 20958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:45.309562 20958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:45.310647 20958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } }
I20260811 07:07:45.310751 20958 leader_election.cc:302] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e74da8cbde244e198b16aaff4a662fe; no voters:
I20260811 07:07:45.310884 20958 leader_election.cc:288] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:45.310968 20972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:45.311152 20972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [term 1 LEADER]: Becoming Leader. State: Replica: 8e74da8cbde244e198b16aaff4a662fe, State: Running, Role: LEADER
I20260811 07:07:45.311209 20958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:45.311260 20972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } }
I20260811 07:07:45.311686 20973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e74da8cbde244e198b16aaff4a662fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } } }
I20260811 07:07:45.311723 20974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e74da8cbde244e198b16aaff4a662fe. Latest consensus state: current_term: 1 leader_uuid: "8e74da8cbde244e198b16aaff4a662fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e74da8cbde244e198b16aaff4a662fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35503 } } }
I20260811 07:07:45.311947 20974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:45.311849 20973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:45.312412 20985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:45.312889 20985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:45.314435 20985 catalog_manager.cc:1347] Generated new cluster ID: a57cea4c0e464324a0c6d9f54127d76f
I20260811 07:07:45.314493 20985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:45.320756 20985 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:45.320915 20985 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:45.333913 20985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e74da8cbde244e198b16aaff4a662fe: Generated new TSK 0
I20260811 07:07:45.334105 20985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:07:45.341042 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--builtin_ntp_servers=127.3.236.148:36425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:07:45.431147 20998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:45.431362 20998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:45.431385 20998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:45.432662 20998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:45.432705 20998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:45.432732 20998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:45.434090 20998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:45.434372 20998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:45.434697 20998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:45.440024 21011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.440032 21019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.440032 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.440506 20998 server_base.cc:1034] running on GCE node
I20260811 07:07:45.440719 20998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:45.440968 20998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:45.442129 20998 hybrid_clock.cc:648] HybridClock initialized: now 1786432065442115 us; error 27 us; skew 500 ppm
I20260811 07:07:45.443610 20998 webserver.cc:460] Webserver started at http://127.3.236.129:32945/ using document root <none> and password file <none>
I20260811 07:07:45.443881 20998 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:45.443936 20998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:45.444067 20998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:45.445586 20998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "c29f68fcbe704303814aa6d4b6d7fe95"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "a8e8e7838e29faa3bfb7c64bc48b4fee"
server_key_iv: "3d3775fa1ec860aca3334c9953aac261"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.446066 20998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "c29f68fcbe704303814aa6d4b6d7fe95"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "a8e8e7838e29faa3bfb7c64bc48b4fee"
server_key_iv: "3d3775fa1ec860aca3334c9953aac261"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.447703 20998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:45.448580 21029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.448791 20998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:07:45.448892 20998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "c29f68fcbe704303814aa6d4b6d7fe95"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "a8e8e7838e29faa3bfb7c64bc48b4fee"
server_key_iv: "3d3775fa1ec860aca3334c9953aac261"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.448963 20998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:45.452587 20998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:45.452816 20998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:45.452911 20998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:45.453063 20998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:45.453308 20998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:45.453337 20998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.453358 20998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:45.453372 20998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.459538 20998 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:34069
I20260811 07:07:45.459602 21212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:34069 every 8 connection(s)
I20260811 07:07:45.459928 20998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:45.463315 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.460894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:56635 (local address 127.3.236.190:35503)
0811 07:07:45.461000 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.461003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.461286 (+ 283us) server_negotiation.cc:407] Connection header received
0811 07:07:45.461772 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.461773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.461778 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.461830 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.463116 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:45.463117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:45.463117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:45.463121 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:45.463133 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:45.463137 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:45.463182 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:45.463247 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 07:07:45.463259 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:07:45.463500 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.460859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35503 (local address 127.3.236.129:56635)
0811 07:07:45.461131 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:45.461176 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:45.461675 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:45.461892 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:45.461896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:45.461904 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:45.462967 (+ 1063us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:45.462968 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:45.462998 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:45.463029 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:45.463183 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:45.463186 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:45.463188 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:07:45.463239 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 07:07:45.463288 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260811 07:07:45.464073 21213 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35503
I20260811 07:07:45.464186 21213 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:45.464366 21213 heartbeater.cc:511] Master 127.3.236.190:35503 requested a full tablet report, sending...
I20260811 07:07:45.464866 20900 ts_manager.cc:194] Registered new tserver with Master: c29f68fcbe704303814aa6d4b6d7fe95 (127.3.236.129:34069)
I20260811 07:07:45.467710 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 20998
I20260811 07:07:45.467823 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:45.468026 4018 external_mini_cluster.cc:1428] Setting key 82c2cda9a403d089959dec61eea165c4
I20260811 07:07:45.468954 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--builtin_ntp_servers=127.3.236.148:36425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:07:45.553462 21221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:45.553680 21221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:45.553704 21221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:45.554914 21221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:45.554956 21221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:45.554981 21221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:45.556295 21221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:45.556483 21221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:45.556725 21221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:45.561932 21229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.561946 21231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.562090 21228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.562393 21221 server_base.cc:1034] running on GCE node
I20260811 07:07:45.562618 21221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:45.562882 21221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:45.564061 21221 hybrid_clock.cc:648] HybridClock initialized: now 1786432065564045 us; error 27 us; skew 500 ppm
I20260811 07:07:45.565353 21221 webserver.cc:460] Webserver started at http://127.3.236.130:39963/ using document root <none> and password file <none>
I20260811 07:07:45.565608 21221 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:45.565681 21221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:45.565841 21221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:45.567073 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "80965f1028234d86a9aba4bf2bdaeb79"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "72053c97045bd47d394d5e58bc5efd11"
server_key_iv: "2b8bf6de9797a0706255e6c541d8e51d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.567512 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "80965f1028234d86a9aba4bf2bdaeb79"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "72053c97045bd47d394d5e58bc5efd11"
server_key_iv: "2b8bf6de9797a0706255e6c541d8e51d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.569232 21221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260811 07:07:45.569979 21242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.570148 21221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:45.570256 21221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "80965f1028234d86a9aba4bf2bdaeb79"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "72053c97045bd47d394d5e58bc5efd11"
server_key_iv: "2b8bf6de9797a0706255e6c541d8e51d"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.570338 21221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:45.573907 21221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:45.574175 21221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:45.574321 21221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:45.574548 21221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:45.574857 21221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:45.574913 21221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.574947 21221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:45.574973 21221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.581225 21221 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41771
I20260811 07:07:45.581266 21439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41771 every 8 connection(s)
I20260811 07:07:45.581634 21221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:45.584738 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 21221
I20260811 07:07:45.584817 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.582445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:41357 (local address 127.3.236.190:35503)
0811 07:07:45.582516 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.582518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.582744 (+ 226us) server_negotiation.cc:407] Connection header received
0811 07:07:45.583214 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.583215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.583220 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.583284 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.584545 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:45.584547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:45.584547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:45.584554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:45.584574 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:45.584580 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:45.584658 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:45.584709 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:07:45.584727 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:07:45.584841 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:45.585028 4018 external_mini_cluster.cc:1428] Setting key 582f16bd2e71fe57136774729674d73b
I20260811 07:07:45.585081 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.582421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35503 (local address 127.3.236.130:41357)
0811 07:07:45.582611 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:45.582652 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:45.583127 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:45.583337 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:45.583340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:45.583348 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:45.584327 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:45.584328 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:45.584365 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:45.584396 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:45.584654 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:45.584656 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:45.584659 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:45.584712 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 07:07:45.584765 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260811 07:07:45.585608 21440 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35503
I20260811 07:07:45.585731 21440 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:45.586119 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--builtin_ntp_servers=127.3.236.148:36425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 07:07:45.587879 21440 heartbeater.cc:511] Master 127.3.236.190:35503 requested a full tablet report, sending...
I20260811 07:07:45.588392 20900 ts_manager.cc:194] Registered new tserver with Master: 80965f1028234d86a9aba4bf2bdaeb79 (127.3.236.130:41771)
W20260811 07:07:45.671819 21448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:45.672087 21448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:45.672129 21448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:45.673470 21448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:45.673534 21448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:45.673578 21448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:45.674997 21448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:45.675273 21448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:45.675571 21448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:07:45.680855 21448 server_base.cc:1034] running on GCE node
W20260811 07:07:45.680786 21458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.680802 21459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:45.680815 21463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:45.681303 21448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:45.681602 21448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:45.682809 21448 hybrid_clock.cc:648] HybridClock initialized: now 1786432065682798 us; error 40 us; skew 500 ppm
I20260811 07:07:45.684243 21448 webserver.cc:460] Webserver started at http://127.3.236.131:43915/ using document root <none> and password file <none>
I20260811 07:07:45.684566 21448 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:45.684638 21448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:45.684804 21448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:45.685946 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "f0fb44cb0ee044a7a91a5dc92e3c3067"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "40a56597445e9bbb3c2a02fb466fab39"
server_key_iv: "14c22eb8ca53506492761a889e65a505"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.686416 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "f0fb44cb0ee044a7a91a5dc92e3c3067"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "40a56597445e9bbb3c2a02fb466fab39"
server_key_iv: "14c22eb8ca53506492761a889e65a505"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.687805 21448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:07:45.688645 21473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.688812 21448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.688953 21448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f0fb44cb0ee044a7a91a5dc92e3c3067"
format_stamp: "Formatted at 2026-08-11 07:07:45 on dist-test-slave-2d19"
server_key: "40a56597445e9bbb3c2a02fb466fab39"
server_key_iv: "14c22eb8ca53506492761a889e65a505"
server_key_version: "encryptionkey@0"
I20260811 07:07:45.689035 21448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:45.692402 21448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:45.692667 21448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:45.692808 21448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:45.693034 21448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:45.693343 21448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:45.693396 21448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.693428 21448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:45.693454 21448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:45.699676 21448 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:44165
I20260811 07:07:45.699774 21674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:44165 every 8 connection(s)
I20260811 07:07:45.700129 21448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:45.701967 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 21448
I20260811 07:07:45.702055 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:45.702302 4018 external_mini_cluster.cc:1428] Setting key 6a8f4fbd6e74b191160028d16c458113
I20260811 07:07:45.703740 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.701012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:41139 (local address 127.3.236.190:35503)
0811 07:07:45.701066 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.701069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.701294 (+ 225us) server_negotiation.cc:407] Connection header received
0811 07:07:45.701809 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.701811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.701818 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.701896 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.703441 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:45.703443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:45.703443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:07:45.703448 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:45.703465 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:07:45.703471 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:45.703531 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:45.703647 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 07:07:45.703659 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:07:45.703984 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:45.700961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35503 (local address 127.3.236.131:41139)
0811 07:07:45.701171 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:45.701204 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:45.701646 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:45.701967 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:45.701971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:45.701979 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:45.703255 (+ 1276us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:07:45.703256 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:45.703286 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:07:45.703352 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:45.703574 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:45.703586 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:45.703589 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:45.703658 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:07:45.703721 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260811 07:07:45.704460 21675 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35503
I20260811 07:07:45.704546 21675 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:45.704725 21675 heartbeater.cc:511] Master 127.3.236.190:35503 requested a full tablet report, sending...
I20260811 07:07:45.705125 20900 ts_manager.cc:194] Registered new tserver with Master: f0fb44cb0ee044a7a91a5dc92e3c3067 (127.3.236.131:44165)
I20260811 07:07:45.714334 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:07:45.716187 20967 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:07:45.715742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.3.236.190:35503)
0811 07:07:45.715795 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:45.715797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:45.715901 (+ 104us) server_negotiation.cc:407] Connection header received
0811 07:07:45.715966 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:45.715966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:45.715971 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:45.716007 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:45.716133 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35460: BlockingRecv error: recv got EOF from 127.0.0.1:35460 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:07:45.716275 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 20998
I20260811 07:07:45.729717 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 21221
I20260811 07:07:45.740782 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 21448
I20260811 07:07:45.751813 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 20837
2026-08-11T07:07:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (600 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:45 dist-test-slave-2d19 krb5kdc[21695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:45 dist-test-slave-2d19 krb5kdc[21695](info): set up 2 sockets
Aug 11 07:07:45 dist-test-slave-2d19 krb5kdc[21695](info): commencing operation
krb5kdc: starting...
W20260811 07:07:47.057638 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:07:47.183641 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36327
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:36327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:47.268888 21718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:47.269124 21718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:47.269160 21718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:47.270287 21718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:07:47.270351 21718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:47.270375 21718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:47.270397 21718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:07:47.270423 21718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:07:47.271937 21718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:36327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:47.272212 21718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:47.272454 21718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:47.277303 21733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.277275 21731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.277277 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:47.277572 21718 server_base.cc:1034] running on GCE node
I20260811 07:07:47.277809 21718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:47.278126 21718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:47.279286 21718 hybrid_clock.cc:648] HybridClock initialized: now 1786432067279266 us; error 35 us; skew 500 ppm
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:47.281761 21718 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:07:47.282217 21718 webserver.cc:460] Webserver started at http://127.3.236.190:36029/ using document root <none> and password file <none>
I20260811 07:07:47.282440 21718 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:47.282539 21718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:47.282688 21718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:47.283710 21718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "a67053d66ee6499485dcafe27ed59bad"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "9bac1c0ddff7d0d32d491a117ad5ead7"
server_key_iv: "285d037ad85f659d0de34d8d6235a5ca"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.284080 21718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "a67053d66ee6499485dcafe27ed59bad"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "9bac1c0ddff7d0d32d491a117ad5ead7"
server_key_iv: "285d037ad85f659d0de34d8d6235a5ca"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.285468 21718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:47.286175 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.286343 21718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.286441 21718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "a67053d66ee6499485dcafe27ed59bad"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "9bac1c0ddff7d0d32d491a117ad5ead7"
server_key_iv: "285d037ad85f659d0de34d8d6235a5ca"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.286535 21718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:47.298374 21718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:47.300172 21718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:47.300345 21718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:47.304461 21718 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:36327
I20260811 07:07:47.304492 21836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:36327 every 8 connection(s)
I20260811 07:07:47.304854 21718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:07:47.305898 21838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:47.308151 21838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: Bootstrap starting.
I20260811 07:07:47.308712 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 21718
I20260811 07:07:47.308813 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:07:47.308919 21838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:47.309028 4018 external_mini_cluster.cc:1428] Setting key b1863627f5ddfaf90763303b50ffc0fd
I20260811 07:07:47.309253 21838 log.cc:824] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:47.309993 21838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: No bootstrap required, opened a new log
I20260811 07:07:47.311861 21838 raft_consensus.cc:348] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } }
I20260811 07:07:47.312005 21838 raft_consensus.cc:374] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:47.312027 21838 raft_consensus.cc:729] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a67053d66ee6499485dcafe27ed59bad, State: Initialized, Role: FOLLOWER
I20260811 07:07:47.312177 21838 consensus_queue.cc:260] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } }
I20260811 07:07:47.312311 21838 raft_consensus.cc:388] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:07:47.312359 21838 raft_consensus.cc:482] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:07:47.312409 21838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:47.312922 21838 raft_consensus.cc:504] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } }
I20260811 07:07:47.313037 21838 leader_election.cc:302] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a67053d66ee6499485dcafe27ed59bad; no voters:
I20260811 07:07:47.313231 21838 leader_election.cc:288] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:07:47.313254 21844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:47.313369 21844 raft_consensus.cc:686] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [term 1 LEADER]: Becoming Leader. State: Replica: a67053d66ee6499485dcafe27ed59bad, State: Running, Role: LEADER
I20260811 07:07:47.313499 21838 sys_catalog.cc:564] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:07:47.313514 21844 consensus_queue.cc:237] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } }
I20260811 07:07:47.314201 21844 sys_catalog.cc:455] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a67053d66ee6499485dcafe27ed59bad. Latest consensus state: current_term: 1 leader_uuid: "a67053d66ee6499485dcafe27ed59bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } } }
I20260811 07:07:47.314304 21844 sys_catalog.cc:458] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [sys.catalog]: This master's current role is: LEADER
I20260811 07:07:47.314687 21858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:07:47.315200 21858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:07:47.315274 21846 sys_catalog.cc:455] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a67053d66ee6499485dcafe27ed59bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67053d66ee6499485dcafe27ed59bad" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36327 } } }
I20260811 07:07:47.315347 21846 sys_catalog.cc:458] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad [sys.catalog]: This master's current role is: LEADER
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:47.316903 21858 catalog_manager.cc:1347] Generated new cluster ID: e6d3f846aafe426a96f10ba83cd6c139
I20260811 07:07:47.316959 21858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:07:47.319716 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.309561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.3.236.190:36327)
0811 07:07:47.309740 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.309749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.309785 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:07:47.310279 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.310286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.310334 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.310471 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:47.311140 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.311998 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.312895 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.313459 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.313606 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.316536 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:47.316540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:47.316540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:47.316551 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:47.318221 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.318581 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.318584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.318585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.318682 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.318887 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.318888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.318888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.319126 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:47.319329 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.319410 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.319545 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260811 07:07:47.361678 21858 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:07:47.362262 21858 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:07:47.364658 21858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: Generated new TSK 0
I20260811 07:07:47.364861 21858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:07:47.401448 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:47.485921 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:47.486174 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:47.486218 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:47.487380 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:47.487445 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:47.487490 21885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:07:47.489048 21885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:47.489329 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:47.489576 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:47.494800 21901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.494801 21905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.494910 21902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:47.494958 21885 server_base.cc:1034] running on GCE node
I20260811 07:07:47.495298 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:47.495569 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:47.496750 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786432067496735 us; error 43 us; skew 500 ppm
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:47.499445 21885 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:07:47.499833 21885 webserver.cc:460] Webserver started at http://127.3.236.129:45265/ using document root <none> and password file <none>
I20260811 07:07:47.500028 21885 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:47.500080 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:47.500175 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:47.501251 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "d736d3a11afe466bb31bedce9c432d94"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "fd575b4ebd557a7f0475e262cba4e839"
server_key_iv: "b391d676e5d8dda7c107c335765da79d"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.501575 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "d736d3a11afe466bb31bedce9c432d94"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "fd575b4ebd557a7f0475e262cba4e839"
server_key_iv: "b391d676e5d8dda7c107c335765da79d"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.502848 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:07:47.503564 21917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.503760 21885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:47.503834 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "d736d3a11afe466bb31bedce9c432d94"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "fd575b4ebd557a7f0475e262cba4e839"
server_key_iv: "b391d676e5d8dda7c107c335765da79d"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.503921 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:47.527204 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:47.529564 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:47.529714 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:47.529979 21885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:47.530311 21885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:47.530354 21885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.530392 21885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:47.530421 21885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.536309 21885 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:42127
I20260811 07:07:47.536371 22107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:42127 every 8 connection(s)
I20260811 07:07:47.536751 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:07:47.537117 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 21885
I20260811 07:07:47.537211 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:07:47.537429 4018 external_mini_cluster.cc:1428] Setting key d77d7164977f50552e5fc848e18ec213
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:07:47.547366 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.537727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:50415 (local address 127.3.236.190:36327)
0811 07:07:47.537778 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.537780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.537856 (+ 76us) server_negotiation.cc:407] Connection header received
0811 07:07:47.538444 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.538446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.538459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.538529 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:47.539441 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.540177 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.541691 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.541902 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.541965 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.544468 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:47.544470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:47.544470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:47.544478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:47.545916 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.546349 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.546350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.546351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.546410 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.546663 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.546664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.546665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.546828 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:47.546896 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.547181 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.547274 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:07:47.547560 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.537555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36327 (local address 127.3.236.129:50415)
0811 07:07:47.537727 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.537770 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.538349 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.538587 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.538590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.539161 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:47.539351 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.539355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.540287 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.540288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.541611 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.541615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.541987 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.541988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.542146 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.542648 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:47.542659 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:47.544367 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:47.546025 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.546033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.546035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.546261 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.546477 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.546477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.546478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.546591 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.546901 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:47.546909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:47.547047 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.547148 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.547308 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260811 07:07:47.548095 22109 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36327
I20260811 07:07:47.548184 22109 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:47.548404 22109 heartbeater.cc:511] Master 127.3.236.190:36327 requested a full tablet report, sending...
I20260811 07:07:47.548907 21782 ts_manager.cc:194] Registered new tserver with Master: d736d3a11afe466bb31bedce9c432d94 (127.3.236.129:42127)
I20260811 07:07:47.549556 21782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:50415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:07:47.567140 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:47.650645 22116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:47.650867 22116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:47.650909 22116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:47.652123 22116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:47.652180 22116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:47.652227 22116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:07:47.653683 22116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:47.653952 22116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:47.654191 22116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:47.659265 22124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.659315 22126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:47.659475 22116 server_base.cc:1034] running on GCE node
W20260811 07:07:47.659511 22128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:47.659734 22116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:47.660010 22116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:47.661178 22116 hybrid_clock.cc:648] HybridClock initialized: now 1786432067661167 us; error 29 us; skew 500 ppm
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:47.663686 22116 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:07:47.664156 22116 webserver.cc:460] Webserver started at http://127.3.236.130:38157/ using document root <none> and password file <none>
I20260811 07:07:47.664439 22116 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:47.664510 22116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:47.664662 22116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:47.665715 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "b8dc33f261634a2f840c9fed936d24b7"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "5209417512a18a831b1f5d56198b9413"
server_key_iv: "6785217e1d967dc18820af75dbca5718"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.666082 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "b8dc33f261634a2f840c9fed936d24b7"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "5209417512a18a831b1f5d56198b9413"
server_key_iv: "6785217e1d967dc18820af75dbca5718"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.667522 22116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:07:47.668287 22140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.668471 22116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:07:47.668570 22116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "b8dc33f261634a2f840c9fed936d24b7"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "5209417512a18a831b1f5d56198b9413"
server_key_iv: "6785217e1d967dc18820af75dbca5718"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.668653 22116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:47.690667 22116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:47.692692 22116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:47.692845 22116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:47.693084 22116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:47.693392 22116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:47.693450 22116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.693492 22116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:47.693523 22116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.699430 22116 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:41109
I20260811 07:07:47.699551 22366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:41109 every 8 connection(s)
I20260811 07:07:47.699820 22116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:07:47.702880 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 22116
I20260811 07:07:47.702991 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:07:47.703207 4018 external_mini_cluster.cc:1428] Setting key 78236b5f388ba0a93135777c33a1be39
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:47.710726 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.700578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47195 (local address 127.3.236.190:36327)
0811 07:07:47.700645 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.700647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.700854 (+ 207us) server_negotiation.cc:407] Connection header received
0811 07:07:47.701394 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.701396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.701408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.701470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:47.702551 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.703358 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.704826 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.705020 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.705074 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.707792 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:47.707794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:47.707794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:47.707802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:47.709322 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.709743 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.709745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.709745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.709806 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.710076 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.710077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.710078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.710190 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:47.710264 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.710551 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.710633 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:07:47.710932 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.700555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36327 (local address 127.3.236.130:47195)
0811 07:07:47.700732 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.700769 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.701288 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.701559 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.701563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.702133 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:47.702333 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.702338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.703462 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.703464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.704729 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.704733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.705102 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.705103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.705254 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.705771 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:47.705783 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:47.707692 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:47.709416 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.709424 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.709427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.709664 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.709874 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.709875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.709876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.709995 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.710255 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:47.710263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:47.710411 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.710514 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.710677 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260811 07:07:47.711416 22367 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36327
I20260811 07:07:47.711540 22367 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:47.711756 22367 heartbeater.cc:511] Master 127.3.236.190:36327 requested a full tablet report, sending...
I20260811 07:07:47.712193 21782 ts_manager.cc:194] Registered new tserver with Master: b8dc33f261634a2f840c9fed936d24b7 (127.3.236.130:41109)
I20260811 07:07:47.712611 21782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:47195
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:07:47.732582 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:07:47.817327 22374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:07:47.817523 22374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:07:47.817548 22374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:07:47.818622 22374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:07:47.818657 22374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:07:47.818683 22374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:07:47.820045 22374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:07:47.820235 22374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:07:47.820484 22374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:07:47.825340 22389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.825430 22386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:07:47.825336 22385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:07:47.825704 22374 server_base.cc:1034] running on GCE node
I20260811 07:07:47.825920 22374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:07:47.826192 22374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:07:47.827334 22374 hybrid_clock.cc:648] HybridClock initialized: now 1786432067827313 us; error 37 us; skew 500 ppm
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:07:47.829845 22374 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:07:47.830341 22374 webserver.cc:460] Webserver started at http://127.3.236.131:41023/ using document root <none> and password file <none>
I20260811 07:07:47.830579 22374 fs_manager.cc:359] Metadata directory not provided
I20260811 07:07:47.830646 22374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:07:47.830788 22374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:07:47.831804 22374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "894410b84bb34473bebab691029b52aa"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "22194ee84793c78d238eb676334e1bcc"
server_key_iv: "37243b7ee476f56b9d505035acb0dcbd"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.832180 22374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "894410b84bb34473bebab691029b52aa"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "22194ee84793c78d238eb676334e1bcc"
server_key_iv: "37243b7ee476f56b9d505035acb0dcbd"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.833650 22374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:07:47.834436 22401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.834606 22374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.834703 22374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "894410b84bb34473bebab691029b52aa"
format_stamp: "Formatted at 2026-08-11 07:07:47 on dist-test-slave-2d19"
server_key: "22194ee84793c78d238eb676334e1bcc"
server_key_iv: "37243b7ee476f56b9d505035acb0dcbd"
server_key_version: "encryptionkey@0"
I20260811 07:07:47.834779 22374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:07:47.853138 22374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:07:47.855293 22374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:07:47.855453 22374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:07:47.855708 22374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:07:47.856037 22374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:07:47.856088 22374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.856132 22374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:07:47.856171 22374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:07:47.862707 22374 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:44971
I20260811 07:07:47.862771 22583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:44971 every 8 connection(s)
I20260811 07:07:47.863082 22374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:07:47.867871 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 22374
I20260811 07:07:47.867981 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:07:47.868206 4018 external_mini_cluster.cc:1428] Setting key 083364c26db9eda709a49c5c196431e6
Aug 11 07:07:47 dist-test-slave-2d19 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432067, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:07:47.872571 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.863825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52931 (local address 127.3.236.190:36327)
0811 07:07:47.863890 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.863892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.864102 (+ 210us) server_negotiation.cc:407] Connection header received
0811 07:07:47.864749 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.864750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.864769 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.864826 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:47.865787 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.866295 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.867297 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.867497 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.867568 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.869868 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:47.869870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:47.869870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:47.869878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:47.871061 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.871508 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.871509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.871509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.871572 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.871843 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.871844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.871844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.871977 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:47.872052 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.872390 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.872472 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:07:47.872777 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.863788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36327 (local address 127.3.236.131:52931)
0811 07:07:47.863984 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.864021 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.864625 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.864897 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.864901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.865490 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:07:47.865684 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.865689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.866398 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.866399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.867209 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.867211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.867591 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.867592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.867778 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.868340 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:07:47.868350 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:07:47.869774 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:07:47.871146 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.871154 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.871156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.871416 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.871654 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:07:47.871655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:07:47.871655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:07:47.871770 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:07:47.872043 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:07:47.872051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:07:47.872192 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.872365 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.872544 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260811 07:07:47.873260 22584 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36327
I20260811 07:07:47.873355 22584 heartbeater.cc:464] Registering TS with master...
I20260811 07:07:47.873536 22584 heartbeater.cc:511] Master 127.3.236.190:36327 requested a full tablet report, sending...
I20260811 07:07:47.873896 21782 ts_manager.cc:194] Registered new tserver with Master: 894410b84bb34473bebab691029b52aa (127.3.236.131:44971)
I20260811 07:07:47.874223 21782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:52931
I20260811 07:07:47.879861 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:07:47.886994 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.881208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.3.236.190:36327)
0811 07:07:47.881271 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.881273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.881339 (+ 66us) server_negotiation.cc:407] Connection header received
0811 07:07:47.881380 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.881381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.881389 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.881435 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:07:47.881931 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.882391 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.883094 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.883267 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.883316 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.884553 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:07:47.884554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:07:47.884555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:07:47.884560 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:07:47.885931 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.886242 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.886243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.886243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.886297 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:07:47.886432 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:07:47.886432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:07:47.886433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:07:47.886564 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:07:47.886642 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.886751 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.886815 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:07:47.888962 21782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:07:47.890806 21782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:07:47.897729 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41109 (local address 127.0.0.1:51956)
0811 07:07:47.893603 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.893645 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.893716 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.894134 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.894137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.894140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:47.894201 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.894202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895545 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.895546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.896679 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.896682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.897444 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.897539 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.897553 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.897599 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.897651 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260811 07:07:47.897948 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:42127 (local address 127.0.0.1:43850)
0811 07:07:47.894617 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.894626 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.894689 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.894867 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.894867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.894869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:47.894918 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.894919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895899 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.895900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.897079 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.897080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.897722 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.897784 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.897788 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.897832 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.897882 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":70,"threads_started":1}
I20260811 07:07:47.898155 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.3.236.130:41109)
0811 07:07:47.893831 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.893836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.893845 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:07:47.893888 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.893890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.893906 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.894025 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:47.894315 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895441 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.896763 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.897346 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897925 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.897983 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.898009 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.898054 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":108,"threads_started":1}
I20260811 07:07:47.898283 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.3.236.131:44971)
0811 07:07:47.894376 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.894380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.894388 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:07:47.894419 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.894420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.894432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.894522 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:47.894685 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895682 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897118 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.897818 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897890 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.897948 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.898140 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.898183 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":105,"threads_started":1}
I20260811 07:07:47.898414 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:44971 (local address 127.0.0.1:54314)
0811 07:07:47.893988 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.893998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.894074 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.894552 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.894553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.894555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:47.894596 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.894598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895781 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.895782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.897028 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.897031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.898000 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.898001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.898060 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.898063 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.898120 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.898157 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":47,"thread_start_us":79,"threads_started":1}
I20260811 07:07:47.898900 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.893992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.3.236.129:42127)
0811 07:07:47.894201 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.894206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.894685 (+ 479us) server_negotiation.cc:407] Connection header received
0811 07:07:47.894767 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.894769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.894783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.894869 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:47.894990 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.895799 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897165 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.897626 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.897708 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.897769 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.898493 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.898541 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260811 07:07:47.898909 22230 tablet_service.cc:1467] Processing CreateTablet for tablet a847473a211742d5b0352178a545e06d (DEFAULT_TABLE table=test-table [id=478858e1a78b42f78cc4519b80a503dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:47.899327 22230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847473a211742d5b0352178a545e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:47.899222 22478 tablet_service.cc:1467] Processing CreateTablet for tablet a847473a211742d5b0352178a545e06d (DEFAULT_TABLE table=test-table [id=478858e1a78b42f78cc4519b80a503dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:47.899590 22478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847473a211742d5b0352178a545e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:47.899575 22001 tablet_service.cc:1467] Processing CreateTablet for tablet a847473a211742d5b0352178a545e06d (DEFAULT_TABLE table=test-table [id=478858e1a78b42f78cc4519b80a503dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:07:47.899899 22001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847473a211742d5b0352178a545e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:07:47.902329 22615 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Bootstrap starting.
I20260811 07:07:47.902922 22618 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Bootstrap starting.
I20260811 07:07:47.903117 22620 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: Bootstrap starting.
I20260811 07:07:47.903503 22615 tablet_bootstrap.cc:654] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:47.903662 22618 tablet_bootstrap.cc:654] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:47.903805 22620 tablet_bootstrap.cc:654] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: Neither blocks nor log segments found. Creating new log.
I20260811 07:07:47.904362 22620 log.cc:824] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:47.904364 22618 log.cc:824] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:47.904364 22615 log.cc:824] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:07:47.905196 22620 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: No bootstrap required, opened a new log
I20260811 07:07:47.905283 22620 ts_tablet_manager.cc:1397] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:07:47.905345 22618 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: No bootstrap required, opened a new log
I20260811 07:07:47.905400 22615 tablet_bootstrap.cc:492] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: No bootstrap required, opened a new log
I20260811 07:07:47.905421 22618 ts_tablet_manager.cc:1397] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:07:47.905463 22615 ts_tablet_manager.cc:1397] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 07:07:47.906895 22620 raft_consensus.cc:348] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.907049 22620 raft_consensus.cc:374] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:47.907092 22620 raft_consensus.cc:729] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d736d3a11afe466bb31bedce9c432d94, State: Initialized, Role: FOLLOWER
I20260811 07:07:47.907205 22620 consensus_queue.cc:260] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.907513 22620 ts_tablet_manager.cc:1428] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:07:47.907477 22615 raft_consensus.cc:348] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.907579 22109 heartbeater.cc:503] Master 127.3.236.190:36327 was elected leader, sending a full tablet report...
I20260811 07:07:47.907631 22615 raft_consensus.cc:374] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:47.907673 22615 raft_consensus.cc:729] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8dc33f261634a2f840c9fed936d24b7, State: Initialized, Role: FOLLOWER
I20260811 07:07:47.907807 22615 consensus_queue.cc:260] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.907418 22618 raft_consensus.cc:348] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.907932 22618 raft_consensus.cc:374] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:07:47.907986 22618 raft_consensus.cc:729] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894410b84bb34473bebab691029b52aa, State: Initialized, Role: FOLLOWER
I20260811 07:07:47.908124 22615 ts_tablet_manager.cc:1428] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:07:47.908118 22618 consensus_queue.cc:260] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.908278 22367 heartbeater.cc:503] Master 127.3.236.190:36327 was elected leader, sending a full tablet report...
I20260811 07:07:47.908416 22618 ts_tablet_manager.cc:1428] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:07:47.908438 22584 heartbeater.cc:503] Master 127.3.236.190:36327 was elected leader, sending a full tablet report...
I20260811 07:07:47.928514 22628 raft_consensus.cc:482] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:07:47.928660 22628 raft_consensus.cc:504] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.929111 22628 leader_election.cc:288] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8dc33f261634a2f840c9fed936d24b7 (127.3.236.130:41109), 894410b84bb34473bebab691029b52aa (127.3.236.131:44971)
I20260811 07:07:47.932992 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.929484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:35599 (local address 127.3.236.131:44971)
0811 07:07:47.929556 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.929558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.929618 (+ 60us) server_negotiation.cc:407] Connection header received
0811 07:07:47.929675 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.929676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.929683 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.929745 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:47.929867 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.930549 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932031 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.932634 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932708 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.932717 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.932887 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.932924 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:07:47.932992 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.929350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:46681 (local address 127.3.236.130:41109)
0811 07:07:47.929412 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.929415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.929425 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:07:47.929452 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.929453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.929464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.929543 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:07:47.929726 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.930504 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932031 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.932628 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932709 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.932717 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.932887 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.932924 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 07:07:47.933094 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.929272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:41109 (local address 127.3.236.129:46681)
0811 07:07:47.929328 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.929341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.929383 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.929577 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.929578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.929580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:47.929659 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.929660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.930612 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.930613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.931933 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.931937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.932717 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.932810 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.932811 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.932872 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.932918 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":53}
I20260811 07:07:47.933301 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.929393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:44971 (local address 127.3.236.129:35599)
0811 07:07:47.929561 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.929571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:07:47.929626 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:07:47.929760 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:07:47.929761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:07:47.929762 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:07:47.929809 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.929810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.930621 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.930622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.931933 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:07:47.931937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.932717 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.932718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:07:47.932806 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.932811 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:07:47.932872 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:07:47.932918 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":84,"thread_start_us":123,"threads_started":1}
I20260811 07:07:47.933425 22265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847473a211742d5b0352178a545e06d" candidate_uuid: "d736d3a11afe466bb31bedce9c432d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8dc33f261634a2f840c9fed936d24b7" is_pre_election: true
I20260811 07:07:47.933597 22265 raft_consensus.cc:2454] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d736d3a11afe466bb31bedce9c432d94 in term 0.
I20260811 07:07:47.933616 22508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847473a211742d5b0352178a545e06d" candidate_uuid: "d736d3a11afe466bb31bedce9c432d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894410b84bb34473bebab691029b52aa" is_pre_election: true
I20260811 07:07:47.933789 22508 raft_consensus.cc:2454] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d736d3a11afe466bb31bedce9c432d94 in term 0.
I20260811 07:07:47.933820 21919 leader_election.cc:302] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8dc33f261634a2f840c9fed936d24b7, d736d3a11afe466bb31bedce9c432d94; no voters:
I20260811 07:07:47.934054 22628 raft_consensus.cc:2781] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:07:47.934106 22628 raft_consensus.cc:482] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:07:47.934127 22628 raft_consensus.cc:3037] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:47.934914 22628 raft_consensus.cc:504] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.935043 22628 leader_election.cc:288] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [CANDIDATE]: Term 1 election: Requested vote from peers b8dc33f261634a2f840c9fed936d24b7 (127.3.236.130:41109), 894410b84bb34473bebab691029b52aa (127.3.236.131:44971)
I20260811 07:07:47.935231 22265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847473a211742d5b0352178a545e06d" candidate_uuid: "d736d3a11afe466bb31bedce9c432d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8dc33f261634a2f840c9fed936d24b7"
I20260811 07:07:47.935247 22508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847473a211742d5b0352178a545e06d" candidate_uuid: "d736d3a11afe466bb31bedce9c432d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894410b84bb34473bebab691029b52aa"
I20260811 07:07:47.935303 22265 raft_consensus.cc:3037] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:47.935338 22508 raft_consensus.cc:3037] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:07:47.935948 22265 raft_consensus.cc:2454] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d736d3a11afe466bb31bedce9c432d94 in term 1.
I20260811 07:07:47.936106 21919 leader_election.cc:302] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8dc33f261634a2f840c9fed936d24b7, d736d3a11afe466bb31bedce9c432d94; no voters:
I20260811 07:07:47.936174 22508 raft_consensus.cc:2454] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d736d3a11afe466bb31bedce9c432d94 in term 1.
I20260811 07:07:47.936224 22628 raft_consensus.cc:2781] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:07:47.936426 22628 raft_consensus.cc:686] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [term 1 LEADER]: Becoming Leader. State: Replica: d736d3a11afe466bb31bedce9c432d94, State: Running, Role: LEADER
I20260811 07:07:47.936528 22628 consensus_queue.cc:237] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } }
I20260811 07:07:47.937444 21777 catalog_manager.cc:5495] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to d736d3a11afe466bb31bedce9c432d94 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "d736d3a11afe466bb31bedce9c432d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d736d3a11afe466bb31bedce9c432d94" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42127 } health_report { overall_health: HEALTHY } } }
I20260811 07:07:47.942777 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.940430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.3.236.129:42127)
0811 07:07:47.940469 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 07:07:47.940471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:07:47.940510 (+ 39us) server_negotiation.cc:407] Connection header received
0811 07:07:47.940584 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:07:47.940585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:07:47.940600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:07:47.940661 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:07:47.940774 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.941224 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.942002 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:07:47.942180 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:07:47.942231 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:07:47.942433 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:07:47.942574 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:07:47.942656 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:07:47.942689 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 07:07:47.942706 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:07:47.944660 22508 raft_consensus.cc:1261] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 1 FOLLOWER]: Refusing update from remote peer d736d3a11afe466bb31bedce9c432d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:47.944667 22265 raft_consensus.cc:1261] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 1 FOLLOWER]: Refusing update from remote peer d736d3a11afe466bb31bedce9c432d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:07:47.945103 22640 consensus_queue.cc:1035] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8dc33f261634a2f840c9fed936d24b7" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:47.945223 22628 consensus_queue.cc:1035] T a847473a211742d5b0352178a545e06d P d736d3a11afe466bb31bedce9c432d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "894410b84bb34473bebab691029b52aa" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 44971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:07:47.950325 21777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:47.950486 21777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:07:47.951349 21777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a67053d66ee6499485dcafe27ed59bad: Sending DeleteTablet for 3 replicas of tablet a847473a211742d5b0352178a545e06d
I20260811 07:07:47.951743 22230 tablet_service.cc:1514] Processing DeleteTablet for tablet a847473a211742d5b0352178a545e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:47 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:51956
I20260811 07:07:47.951864 22478 tablet_service.cc:1514] Processing DeleteTablet for tablet a847473a211742d5b0352178a545e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:47 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:54314
I20260811 07:07:47.952121 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 21885
I20260811 07:07:47.952111 22001 tablet_service.cc:1514] Processing DeleteTablet for tablet a847473a211742d5b0352178a545e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:07:47 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:43850
I20260811 07:07:47.952142 22668 tablet_replica.cc:330] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: stopping tablet replica
I20260811 07:07:47.952713 22668 raft_consensus.cc:2229] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:47.952827 22668 raft_consensus.cc:2258] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:47.958251 22668 ts_tablet_manager.cc:1905] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:47.961705 22668 ts_tablet_manager.cc:1918] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:47.961789 22668 log.cc:1196] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/a847473a211742d5b0352178a545e06d
I20260811 07:07:47.962069 22668 ts_tablet_manager.cc:1939] T a847473a211742d5b0352178a545e06d P 894410b84bb34473bebab691029b52aa: Deleting consensus metadata
I20260811 07:07:47.962525 21753 catalog_manager.cc:4841] TS 894410b84bb34473bebab691029b52aa (127.3.236.131:44971): tablet a847473a211742d5b0352178a545e06d (table test-table [id=478858e1a78b42f78cc4519b80a503dd]) successfully deleted
I20260811 07:07:47.962944 22667 tablet_replica.cc:330] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: stopping tablet replica
I20260811 07:07:47.963050 22667 raft_consensus.cc:2229] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:07:47.963191 22667 raft_consensus.cc:2258] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:07:47.963709 22667 ts_tablet_manager.cc:1905] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:07:47.965245 22667 ts_tablet_manager.cc:1918] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:07:47.965345 22667 log.cc:1196] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/a847473a211742d5b0352178a545e06d
I20260811 07:07:47.965649 22667 ts_tablet_manager.cc:1939] T a847473a211742d5b0352178a545e06d P b8dc33f261634a2f840c9fed936d24b7: Deleting consensus metadata
I20260811 07:07:47.966105 21753 catalog_manager.cc:4841] TS b8dc33f261634a2f840c9fed936d24b7 (127.3.236.130:41109): tablet a847473a211742d5b0352178a545e06d (table test-table [id=478858e1a78b42f78cc4519b80a503dd]) successfully deleted
W20260811 07:07:47.968032 21753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:42127 (error 108)
I20260811 07:07:47.968624 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:07:47.968480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:42127 (local address 127.0.0.1:43874)
0811 07:07:47.968541 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 07:07:47.968586 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:42127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 07:07:47.968705 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 22116
W20260811 07:07:47.968751 21753 catalog_manager.cc:4584] TS d736d3a11afe466bb31bedce9c432d94 (127.3.236.129:42127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a847473a211742d5b0352178a545e06d: Network error: Client connection negotiation failed: client connection to 127.3.236.129:42127: connect: Connection refused (error 111)
I20260811 07:07:48.005036 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 22374
I20260811 07:07:48.018464 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 21718
2026-08-11T07:07:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2265 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:07:48 dist-test-slave-2d19 krb5kdc[22674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:07:48 dist-test-slave-2d19 krb5kdc[22674](info): set up 2 sockets
Aug 11 07:07:48 dist-test-slave-2d19 krb5kdc[22674](info): commencing operation
krb5kdc: starting...
W20260811 07:07:49.309827 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:07:49 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:07:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangeradmin_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangeradmin_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangerlookup_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangerlookup_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
I20260811 07:07:49.483078 4018 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhrK1s3/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/postgres -l logfile start
2026-08-11 07:07:52.023 UTC [22724] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 07:07:52.023 UTC [22724] LOG: listening on IPv4 address "127.3.236.148", port 51199
2026-08-11 07:07:52.027 UTC [22724] LOG: listening on Unix socket "/tmp/.s.PGSQL.51199"
2026-08-11 07:07:52.049 UTC [22731] LOG: database system was shut down at 2026-08-11 07:07:50 UTC
2026-08-11 07:07:52.054 UTC [22724] LOG: database system is ready to accept connections
I20260811 07:07:53.291766 4018 mini_postgres.cc:96] Postgres bound to 51199
2026-08-11 07:07:53.299 UTC [22745] FATAL: database "slave" does not exist
127.3.236.148:51199 - accepting connections
I20260811 07:07:53.300246 4018 mini_ranger.cc:161] Starting Ranger...
I20260811 07:07:53.316335 4018 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 07:07:53.791435 4018 mini_ranger.cc:87] Created ranger Postgres database
I20260811 07:07:53.791512 4018 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin
2026-08-11 07:07:54,102 [I] DB FLAVOR :POSTGRES
2026-08-11 07:07:54,103 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:07:54,103 [I] Checking connection..
2026-08-11 07:07:54,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 07:07:54,379 [I] Checking connection passed.
2026-08-11 07:07:54,379 [I] --------- Verifying version history table ---------
2026-08-11 07:07:54,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:07:54,662 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:07:54,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:07:54,936 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:07:54,936 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 07:07:54,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 07:07:55.207 UTC [22860] WARNING: there is no transaction in progress
2026-08-11 07:07:55,214 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 07:07:55,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:07:55,469 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 07:07:55,470 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:07:55,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 07:07:55,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 07:07:55,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2d19.c.gcp-upstream.internal','N') ;"
2026-08-11 07:07:56,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:07:56,492 [I] Table x_portal_user does not exist in database ranger
2026-08-11 07:07:56,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:07:56,763 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 07:07:56,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:07:57,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:07:57,288 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 07:07:57,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 07:07:58.262 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.278 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.294 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.309 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.328 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.349 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.417 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.424 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.439 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.450 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.461 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.476 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.486 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.492 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.499 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:58.506 UTC [23066] WARNING: there is no transaction in progress
2026-08-11 07:07:59,162 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 07:07:59,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 07:07:59,427 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 07:07:59,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:07:59,694 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 07:07:59,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:07:59,932 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 07:07:59,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:08:00,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:08:00,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2d19.c.gcp-upstream.internal';"
2026-08-11 07:08:00,728 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 07:08:00,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 07:08:00,999 [I] DB_PATCHES have already been applied
I20260811 07:08:01.005139 4018 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhrK1s3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 07:08:01.005231 4018 mini_ranger.cc:193] Using host: 127.3.236.148
I20260811 07:08:01.009359 4018 mini_ranger.cc:227] Ranger admin URL: http://127.3.236.148:40659
Aug 11, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:08:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:08:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 07:08:02.457074 4018 mini_ranger.cc:160] Time spent starting Ranger: real 9.157s user 0.003s sys 0.009s
Aug 11, 2026 7:08:23 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:08:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 7:08:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:08:23 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:08:24 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 07:08:25.288617 4018 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangerkms_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/rangerkms_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 07:08:25 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432105, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 07:08:25.380148 4018 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 07:08:25.391062 4018 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 07:08:26.257296 4018 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 07:08:26.257388 4018 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms
I20260811 07:08:26.257411 4018 mini_ranger_kms.cc:227] Using postgres at 127.3.236.148:51199
2026-08-11 07:08:26,917 [I] DB FLAVOR :POSTGRES
2026-08-11 07:08:26,917 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:08:26,917 [I] Checking connection
2026-08-11 07:08:26,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 07:08:27,172 [I] connection success
2026-08-11 07:08:27,172 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 07:08:27,173 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 07:08:27,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 07:08:27,419 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 07:08:27,420 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:08:27,420 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 07:08:27,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:51199/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 07:08:27,684 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 07:08:28.208415 4018 mini_ranger_kms.cc:310] Created kms service
I20260811 07:08:28.432595 4018 mini_ranger_kms.cc:326] Created kudu user
I20260811 07:08:28.481787 4018 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 07:08:28 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432105, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:08:28.916313 4018 mini_ranger_kms.cc:384] Added ranger policy
I20260811 07:08:28.916431 4018 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 07:08:28.916466 4018 mini_ranger_kms.cc:240] Using host: 127.3.236.148
I20260811 07:08:28.920900 4018 mini_ranger_kms.cc:276] Ranger KMS PID: 23399
I20260811 07:08:28.921025 4018 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.236.148:52257
Aug 11, 2026 7:08:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:08:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:08:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:08:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:08:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:08:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 07:08:31 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786432111, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:08:31.779421 4018 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.399s user 0.015s sys 0.305s
I20260811 07:08:31.779547 4018 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 07:08:31.779636 4018 mini_ranger_kms.cc:401] 127.3.236.148:52257/kms/v1/keys
Aug 11 07:08:31 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432111, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
Aug 11 07:08:32 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432111, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
Aug 11 07:08:34 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:08:34.711505 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46227
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46227
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:52257/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:08:34.807642 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:08:34.807863 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:08:34.807888 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:08:34.809213 23483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:08:34.809262 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:08:34.809278 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:08:34.809293 23483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:08:34.809305 23483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:08:34.811071 23483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.236.148:52257/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46227
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:08:34.811326 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:08:34.811610 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:08:34.817224 23497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:34.817258 23496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:34.817414 23483 server_base.cc:1034] running on GCE node
W20260811 07:08:34.817552 23499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:34.817870 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:08:34.818204 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:08:34.819398 23483 hybrid_clock.cc:648] HybridClock initialized: now 1786432114819364 us; error 50 us; skew 500 ppm
Aug 11 07:08:34 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432114, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:08:34.822379 23483 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:08:34.822883 23483 webserver.cc:460] Webserver started at http://127.3.236.190:36511/ using document root <none> and password file <none>
I20260811 07:08:34.823356 23483 fs_manager.cc:359] Metadata directory not provided
I20260811 07:08:34.823454 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:08:34.823623 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:08:34 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432114, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:08:34.874455 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "fe95654e0721482fa695ba64370b53a6"
format_stamp: "Formatted at 2026-08-11 07:08:34 on dist-test-slave-2d19"
server_key: "c538fecd34b9761b3e005e5121a1d011"
server_key_iv: "05d1913b98e4628aba54dd8c73c9640c"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:34.874948 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "fe95654e0721482fa695ba64370b53a6"
format_stamp: "Formatted at 2026-08-11 07:08:34 on dist-test-slave-2d19"
server_key: "c538fecd34b9761b3e005e5121a1d011"
server_key_iv: "05d1913b98e4628aba54dd8c73c9640c"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:34.876639 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 07:08:34.889830 23511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:34.890134 23483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:08:34.890268 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "fe95654e0721482fa695ba64370b53a6"
format_stamp: "Formatted at 2026-08-11 07:08:34 on dist-test-slave-2d19"
server_key: "c538fecd34b9761b3e005e5121a1d011"
server_key_iv: "05d1913b98e4628aba54dd8c73c9640c"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:34.890407 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:08:34.912189 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:08:34.914201 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:08:34.914382 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:08:34.920454 23483 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:46227
I20260811 07:08:34.920513 23603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:46227 every 8 connection(s)
I20260811 07:08:34.920974 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:08:34.921797 23605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:08:34.924630 23605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: Bootstrap starting.
I20260811 07:08:34.925366 23605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: Neither blocks nor log segments found. Creating new log.
I20260811 07:08:34.925740 23605 log.cc:824] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:08:34.926633 23605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: No bootstrap required, opened a new log
I20260811 07:08:34.928993 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 23483
I20260811 07:08:34.929139 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:08:34.929356 4018 external_mini_cluster.cc:1428] Setting key ef12d4e71e935c31142a747b0b8bfa3b
I20260811 07:08:34.929553 23605 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } }
I20260811 07:08:34.929749 23605 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:08:34.929816 23605 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe95654e0721482fa695ba64370b53a6, State: Initialized, Role: FOLLOWER
I20260811 07:08:34.929996 23605 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } }
I20260811 07:08:34.930087 23605 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:08:34.930157 23605 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:08:34.930225 23605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:08:34.930794 23605 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } }
I20260811 07:08:34.930919 23605 leader_election.cc:302] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe95654e0721482fa695ba64370b53a6; no voters:
I20260811 07:08:34.931138 23605 leader_election.cc:288] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:08:34.931162 23611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:08:34.931298 23611 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [term 1 LEADER]: Becoming Leader. State: Replica: fe95654e0721482fa695ba64370b53a6, State: Running, Role: LEADER
I20260811 07:08:34.931433 23605 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:08:34.931447 23611 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } }
I20260811 07:08:34.931959 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe95654e0721482fa695ba64370b53a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } } }
I20260811 07:08:34.932066 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:08:34.932340 23612 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe95654e0721482fa695ba64370b53a6. Latest consensus state: current_term: 1 leader_uuid: "fe95654e0721482fa695ba64370b53a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe95654e0721482fa695ba64370b53a6" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46227 } } }
I20260811 07:08:34.932474 23612 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:08:34.932646 23619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:08:34.933260 23619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:08:34.933797 23605 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 11 07:08:34 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:08:34.945513 23639 mvcc.cc:205] Tried to move back new op lower bound from 7317225942771384320 to 7317225942760042496. Current Snapshot: MvccSnapshot[applied={T|T < 7317225942771384320}]
I20260811 07:08:34.949985 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:34.930089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.3.236.190:46227)
0811 07:08:34.931805 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:34.931817 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:34.931859 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:08:34.932012 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:34.932021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:34.932090 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:34.933151 (+ 1061us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:08:34.933266 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:34.934654 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:34.936812 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:34.938417 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:34.938924 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:34.945102 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:08:34.945108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:08:34.945108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:08:34.945123 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:08:34.947607 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:34.948085 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:34.948089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:34.948089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:34.948195 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:34.948487 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:34.948488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:34.948488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:34.948826 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:08:34.949483 (+ 657us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:34.949564 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:08:34.949707 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1656,"thread_start_us":87,"threads_started":1}
I20260811 07:08:34.952857 23619 catalog_manager.cc:1347] Generated new cluster ID: a87b146602df4a229236390c497d6760
I20260811 07:08:34.952910 23619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:08:34.964933 23619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:08:34.965762 23619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:08:34.973592 23619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: Generated new TSK 0
I20260811 07:08:34.973887 23619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:08:35.761523 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:52257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:08:35.893126 23684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:08:35.893476 23684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:08:35.893520 23684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:08:35.895242 23684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:08:35.895308 23684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:08:35.895350 23684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:08:35.897630 23684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.236.148:52257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:08:35.897881 23684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:08:35.898170 23684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:08:35.904014 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:35.904382 23696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:35.904417 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:35.904721 23684 server_base.cc:1034] running on GCE node
I20260811 07:08:35.905218 23684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:08:35.905493 23684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:08:35.908751 23684 hybrid_clock.cc:648] HybridClock initialized: now 1786432115908731 us; error 27 us; skew 500 ppm
Aug 11 07:08:35 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:08:35.911612 23684 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:08:35.912139 23684 webserver.cc:460] Webserver started at http://127.3.236.129:45517/ using document root <none> and password file <none>
I20260811 07:08:35.912446 23684 fs_manager.cc:359] Metadata directory not provided
I20260811 07:08:35.912525 23684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:08:35.912668 23684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:08:35 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:08:35.927023 23684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "bbce6983db0e4433afb34936adf6e67e"
format_stamp: "Formatted at 2026-08-11 07:08:35 on dist-test-slave-2d19"
server_key: "0760079cbf792cfabe272cc677cb34e9"
server_key_iv: "48b527202332c05fc4ad4495076d092d"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:35.927469 23684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "bbce6983db0e4433afb34936adf6e67e"
format_stamp: "Formatted at 2026-08-11 07:08:35 on dist-test-slave-2d19"
server_key: "0760079cbf792cfabe272cc677cb34e9"
server_key_iv: "48b527202332c05fc4ad4495076d092d"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:35.931761 23684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 07:08:35.947212 23709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:35.947513 23684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:08:35.947649 23684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "bbce6983db0e4433afb34936adf6e67e"
format_stamp: "Formatted at 2026-08-11 07:08:35 on dist-test-slave-2d19"
server_key: "0760079cbf792cfabe272cc677cb34e9"
server_key_iv: "48b527202332c05fc4ad4495076d092d"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:35.947770 23684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:08:35.979506 23684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:08:35.982110 23684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:08:35.982311 23684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:08:35.982628 23684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:08:35.983063 23684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:08:35.983157 23684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:35.983249 23684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:08:35.983323 23684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:35.991991 23684 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:38881
I20260811 07:08:35.992141 23912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:38881 every 8 connection(s)
I20260811 07:08:35.992508 23684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:08:35.993047 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 23684
I20260811 07:08:35.993150 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:08:35.993355 4018 external_mini_cluster.cc:1428] Setting key 2d4a2db6955306d0940d06ec5de11ec3
Aug 11 07:08:35 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432115, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:08:36.004616 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:35.993691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:36625 (local address 127.3.236.190:46227)
0811 07:08:35.993805 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:35.993807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:35.993884 (+ 77us) server_negotiation.cc:407] Connection header received
0811 07:08:35.994654 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:35.994656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:35.994671 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:35.995713 (+ 1042us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:08:35.995755 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:35.996553 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:35.997699 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:35.997919 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:35.997998 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.000751 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:08:36.000753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:08:36.000753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:08:36.000761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:08:36.002784 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.003345 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.003346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.003347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.003419 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.003758 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.003759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.003759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.003912 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:08:36.003986 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.004394 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.004500 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
I20260811 07:08:36.004880 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:35.993344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46227 (local address 127.3.236.129:36625)
0811 07:08:35.993756 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:35.993808 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:35.994529 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:35.994911 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:35.994916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:35.995254 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:08:35.995609 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:35.995616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:35.996660 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:35.996661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:35.997608 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:35.997611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:35.998007 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:35.998008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:35.998157 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:35.998808 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:08:35.998820 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:08:36.000638 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:08:36.002891 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.002899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.002902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.003249 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.003509 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.003510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.003510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.003672 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.003999 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:08:36.004008 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:08:36.004194 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.004361 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.004595 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":89,"threads_started":1}
I20260811 07:08:36.005707 23913 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46227
I20260811 07:08:36.005824 23913 heartbeater.cc:464] Registering TS with master...
I20260811 07:08:36.006070 23913 heartbeater.cc:511] Master 127.3.236.190:46227 requested a full tablet report, sending...
I20260811 07:08:36.006724 23542 ts_manager.cc:194] Registered new tserver with Master: bbce6983db0e4433afb34936adf6e67e (127.3.236.129:38881)
I20260811 07:08:36.007601 23542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:36625
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:08:36.035303 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:52257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:08:36.171278 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:08:36.171689 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:08:36.171916 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:08:36.173796 23929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:08:36.173915 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:08:36.174009 23929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:08:36.176606 23929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.236.148:52257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:08:36.177042 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:08:36.177381 23929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:08:36.184751 23943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:36.184785 23951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:36.185047 23948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:36.185755 23929 server_base.cc:1034] running on GCE node
I20260811 07:08:36.185992 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:08:36.186314 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:08:36.187526 23929 hybrid_clock.cc:648] HybridClock initialized: now 1786432116187491 us; error 46 us; skew 500 ppm
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:08:36.191246 23929 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:08:36.191869 23929 webserver.cc:460] Webserver started at http://127.3.236.130:41607/ using document root <none> and password file <none>
I20260811 07:08:36.192175 23929 fs_manager.cc:359] Metadata directory not provided
I20260811 07:08:36.192272 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:08:36.192457 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:08:36.207485 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "bb5a95f936e048f78c9d3f944b7f3bce"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "9aecf79c88c8f737298f1f8a4114a81c"
server_key_iv: "1d1ce85b501e92c42673f05cfc06fcd1"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.207953 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "bb5a95f936e048f78c9d3f944b7f3bce"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "9aecf79c88c8f737298f1f8a4114a81c"
server_key_iv: "1d1ce85b501e92c42673f05cfc06fcd1"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.216758 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.001s
I20260811 07:08:36.230458 23962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.230712 23929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:08:36.230818 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "bb5a95f936e048f78c9d3f944b7f3bce"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "9aecf79c88c8f737298f1f8a4114a81c"
server_key_iv: "1d1ce85b501e92c42673f05cfc06fcd1"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.230937 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:08:36.256732 23929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:08:36.267549 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:08:36.267728 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:08:36.267978 23929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:08:36.268419 23929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:08:36.268471 23929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.268501 23929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:08:36.268519 23929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.276923 23929 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:33665
I20260811 07:08:36.277033 24174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:33665 every 8 connection(s)
I20260811 07:08:36.277418 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:08:36.278329 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 23929
I20260811 07:08:36.278435 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:08:36.278646 4018 external_mini_cluster.cc:1428] Setting key b0c6ddb6a2e2dd1d03a535a06b3e8236
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
I20260811 07:08:36.295038 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.278421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:57427 (local address 127.3.236.190:46227)
0811 07:08:36.278506 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.278509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.278805 (+ 296us) server_negotiation.cc:407] Connection header received
0811 07:08:36.279466 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.279468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.279483 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.279512 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:08:36.280166 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.281010 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.282864 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.283284 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.283335 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.291058 (+ 7723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:08:36.291060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:08:36.291060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:08:36.291069 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:08:36.293024 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.293566 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.293568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.293568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.293627 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.294060 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.294061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.294061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.294214 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:08:36.294267 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.294778 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.294897 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:08:36.295831 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.278441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46227 (local address 127.3.236.130:57427)
0811 07:08:36.278610 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.278652 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.279343 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.279673 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.279676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.279868 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:08:36.280069 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.280074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.281130 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.281132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.282745 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.282749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.283457 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.283458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.283643 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.289288 (+ 5645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:08:36.289301 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:08:36.290923 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:08:36.293132 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.293142 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.293145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.293473 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.293708 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.293709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.293790 (+ 81us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.293965 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.294326 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:08:36.294333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:08:36.294593 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.295411 (+ 818us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.295596 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260811 07:08:36.296528 24180 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46227
I20260811 07:08:36.296635 24180 heartbeater.cc:464] Registering TS with master...
I20260811 07:08:36.296834 24180 heartbeater.cc:511] Master 127.3.236.190:46227 requested a full tablet report, sending...
I20260811 07:08:36.297307 23542 ts_manager.cc:194] Registered new tserver with Master: bb5a95f936e048f78c9d3f944b7f3bce (127.3.236.130:33665)
I20260811 07:08:36.297760 23542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:57427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:08:36.315128 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:52257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:08:36.415341 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:08:36.415570 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:08:36.415609 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:08:36.416960 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:08:36.417029 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:08:36.417083 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:08:36.418633 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.236.148:52257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:46227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:08:36.418908 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:08:36.419217 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:08:36.425848 24202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:36.425999 24192 server_base.cc:1034] running on GCE node
W20260811 07:08:36.426131 24209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:08:36.428365 24201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:08:36.428655 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:08:36.428949 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:08:36.430135 24192 hybrid_clock.cc:648] HybridClock initialized: now 1786432116430121 us; error 32 us; skew 500 ppm
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:08:36.433617 24192 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:08:36.434156 24192 webserver.cc:460] Webserver started at http://127.3.236.131:35387/ using document root <none> and password file <none>
I20260811 07:08:36.434458 24192 fs_manager.cc:359] Metadata directory not provided
I20260811 07:08:36.434523 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:08:36.434674 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:08:36.446343 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "1dcdc46b41704e28bff5f43da6cf9cc1"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "aba63bb79dd5008ee586e6d4ad47ca85"
server_key_iv: "ddd95c6a6213303d640ac5a383a187be"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.446834 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "1dcdc46b41704e28bff5f43da6cf9cc1"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "aba63bb79dd5008ee586e6d4ad47ca85"
server_key_iv: "ddd95c6a6213303d640ac5a383a187be"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.451550 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 07:08:36.466354 24218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.472517 24192 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260811 07:08:36.472748 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "1dcdc46b41704e28bff5f43da6cf9cc1"
format_stamp: "Formatted at 2026-08-11 07:08:36 on dist-test-slave-2d19"
server_key: "aba63bb79dd5008ee586e6d4ad47ca85"
server_key_iv: "ddd95c6a6213303d640ac5a383a187be"
server_key_version: "kuduclusterkey@0"
I20260811 07:08:36.472951 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:08:36.495558 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:08:36.498888 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:08:36.499130 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:08:36.499621 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:08:36.500108 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:08:36.500221 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.500346 24192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:08:36.500434 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:08:36.514220 24192 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:45853
I20260811 07:08:36.514700 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:08:36.519477 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 24192
I20260811 07:08:36.519591 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:08:36.519770 4018 external_mini_cluster.cc:1428] Setting key 818c119db7ff2aa4cfacccfe876de0af
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:08:36.523276 24434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:45853 every 8 connection(s)
I20260811 07:08:36.526520 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.516042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38895 (local address 127.3.236.190:46227)
0811 07:08:36.516093 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.516096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.516429 (+ 333us) server_negotiation.cc:407] Connection header received
0811 07:08:36.517115 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.517116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.517129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.517186 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:08:36.517917 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.518696 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.520424 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.520709 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.520745 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.523285 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:08:36.523287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:08:36.523287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:08:36.523295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:08:36.524907 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.525446 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.525447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.525448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.525507 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.525875 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.525875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.525876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.525982 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:08:36.526037 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.526389 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.526455 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:08:36.526768 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.516042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46227 (local address 127.3.236.131:38895)
0811 07:08:36.516218 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.516285 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.516996 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.517263 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.517268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.517511 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:08:36.517751 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.517756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.518829 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.518829 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.520330 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.520333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.520822 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.520823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.520934 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.521525 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:08:36.521534 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:08:36.523156 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:08:36.524996 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.525004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.525007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.525338 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.525589 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:08:36.525590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:08:36.525590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:08:36.525782 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:08:36.526051 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:08:36.526058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:08:36.526231 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.526375 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.526534 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260811 07:08:36.527213 24435 heartbeater.cc:347] Connected to a master server at 127.3.236.190:46227
I20260811 07:08:36.527333 24435 heartbeater.cc:464] Registering TS with master...
I20260811 07:08:36.527536 24435 heartbeater.cc:511] Master 127.3.236.190:46227 requested a full tablet report, sending...
I20260811 07:08:36.527999 23542 ts_manager.cc:194] Registered new tserver with Master: 1dcdc46b41704e28bff5f43da6cf9cc1 (127.3.236.131:45853)
I20260811 07:08:36.528453 23542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:38895
I20260811 07:08:36.531536 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:08:36.540072 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.533267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.3.236.190:46227)
0811 07:08:36.533336 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.533338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.533470 (+ 132us) server_negotiation.cc:407] Connection header received
0811 07:08:36.533523 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.533524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.533536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.533615 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:08:36.534143 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.535043 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.535812 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.536078 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.536176 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.537445 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:08:36.537447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:08:36.537448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:08:36.537456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:08:36.539014 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.539343 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.539344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.539344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.539409 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:08:36.539595 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:08:36.539596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:08:36.539597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:08:36.539724 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:08:36.539824 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.539903 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.539981 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:08:36.542049 23542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:08:36.544586 23542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:08:36.553644 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.547385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:33665 (local address 127.0.0.1:43352)
0811 07:08:36.547912 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.547955 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.548022 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.548504 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.548508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.548511 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:08:36.548585 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.548587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.550193 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.550194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.551769 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.551771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.553253 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.553254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.553337 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.553354 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.553404 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.553466 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":86,"thread_start_us":100,"threads_started":1}
I20260811 07:08:36.554262 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.548007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.3.236.130:33665)
0811 07:08:36.548278 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.548283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.548293 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:08:36.548365 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.548367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.548385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.548663 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:08:36.548745 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.549889 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.552383 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.553151 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.553940 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.554037 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.554063 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.554107 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":147,"threads_started":1}
I20260811 07:08:36.554265 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.547661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38881 (local address 127.0.0.1:34680)
0811 07:08:36.548488 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.548498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.548562 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.548879 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.548880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.548881 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:08:36.548925 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.548927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.550275 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.550276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.552707 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.552711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.554040 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.554041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.554108 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.554112 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.554158 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.554198 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":166,"threads_started":1}
I20260811 07:08:36.554330 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.547543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.3.236.131:45853)
0811 07:08:36.547811 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.547816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.548403 (+ 587us) server_negotiation.cc:407] Connection header received
0811 07:08:36.548697 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.548700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.548716 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.548795 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:08:36.549011 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.550166 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.551975 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.553048 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.553157 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.553330 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.553992 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.554049 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260811 07:08:36.554736 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.547396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:45853 (local address 127.0.0.1:39634)
0811 07:08:36.548328 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.548339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.548618 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.548879 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.548880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.548884 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:08:36.548925 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.548928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.550541 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.550541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.551767 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.551771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.553791 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.553792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.553873 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.553877 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.554511 (+ 634us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.554576 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":16,"thread_start_us":142,"threads_started":1}
I20260811 07:08:36.555045 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.547973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.3.236.129:38881)
0811 07:08:36.548147 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.548152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.548687 (+ 535us) server_negotiation.cc:407] Connection header received
0811 07:08:36.548727 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.548729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.548747 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.548816 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:08:36.549006 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.550168 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.552813 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.553544 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.553715 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.554702 (+ 987us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.554730 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.554779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":57,"threads_started":1}
I20260811 07:08:36.555225 24060 tablet_service.cc:1467] Processing CreateTablet for tablet f9e766c343224dafbf887b2162d3b0b5 (DEFAULT_TABLE table=test-table [id=313c30a6377f4ea880ef582bf5e74f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:08:36.555614 24311 tablet_service.cc:1467] Processing CreateTablet for tablet f9e766c343224dafbf887b2162d3b0b5 (DEFAULT_TABLE table=test-table [id=313c30a6377f4ea880ef582bf5e74f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:08:36.555754 24060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e766c343224dafbf887b2162d3b0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:08:36.555984 24311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e766c343224dafbf887b2162d3b0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:08:36.555884 23799 tablet_service.cc:1467] Processing CreateTablet for tablet f9e766c343224dafbf887b2162d3b0b5 (DEFAULT_TABLE table=test-table [id=313c30a6377f4ea880ef582bf5e74f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:08:36.556211 23799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e766c343224dafbf887b2162d3b0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:08:36.558754 24470 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Bootstrap starting.
I20260811 07:08:36.559878 24471 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Bootstrap starting.
I20260811 07:08:36.560081 24473 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Bootstrap starting.
I20260811 07:08:36.560204 24470 tablet_bootstrap.cc:654] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Neither blocks nor log segments found. Creating new log.
I20260811 07:08:36.560884 24470 log.cc:824] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:08:36.560983 24473 tablet_bootstrap.cc:654] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Neither blocks nor log segments found. Creating new log.
I20260811 07:08:36.561288 24471 tablet_bootstrap.cc:654] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Neither blocks nor log segments found. Creating new log.
I20260811 07:08:36.561429 24473 log.cc:824] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:08:36.561614 24471 log.cc:824] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Log is configured to *not* fsync() on all Append() calls
I20260811 07:08:36.562505 24473 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: No bootstrap required, opened a new log
I20260811 07:08:36.562505 24471 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: No bootstrap required, opened a new log
I20260811 07:08:36.562583 24471 ts_tablet_manager.cc:1397] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:08:36.562583 24473 ts_tablet_manager.cc:1397] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:08:36.564929 24471 raft_consensus.cc:348] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.565212 24471 raft_consensus.cc:374] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:08:36.565311 24471 raft_consensus.cc:729] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5a95f936e048f78c9d3f944b7f3bce, State: Initialized, Role: FOLLOWER
I20260811 07:08:36.565512 24470 tablet_bootstrap.cc:492] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: No bootstrap required, opened a new log
I20260811 07:08:36.565591 24470 ts_tablet_manager.cc:1397] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260811 07:08:36.565554 24473 raft_consensus.cc:348] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.565716 24473 raft_consensus.cc:374] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:08:36.565526 24471 consensus_queue.cc:260] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.565762 24473 raft_consensus.cc:729] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbce6983db0e4433afb34936adf6e67e, State: Initialized, Role: FOLLOWER
I20260811 07:08:36.566079 24471 ts_tablet_manager.cc:1428] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:08:36.566250 24180 heartbeater.cc:503] Master 127.3.236.190:46227 was elected leader, sending a full tablet report...
I20260811 07:08:36.567759 24470 raft_consensus.cc:348] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.567982 24470 raft_consensus.cc:374] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:08:36.568019 24470 raft_consensus.cc:729] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcdc46b41704e28bff5f43da6cf9cc1, State: Initialized, Role: FOLLOWER
I20260811 07:08:36.568136 24470 consensus_queue.cc:260] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.568432 24470 ts_tablet_manager.cc:1428] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:08:36.568483 24435 heartbeater.cc:503] Master 127.3.236.190:46227 was elected leader, sending a full tablet report...
I20260811 07:08:36.580484 24473 consensus_queue.cc:260] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.580972 24473 ts_tablet_manager.cc:1428] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260811 07:08:36.581149 23913 heartbeater.cc:503] Master 127.3.236.190:46227 was elected leader, sending a full tablet report...
Aug 11 07:08:36 dist-test-slave-2d19 krb5kdc[22674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
W20260811 07:08:36.745582 23914 tablet.cc:2369] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:08:36.765609 24437 tablet.cc:2369] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:08:36.778614 24181 tablet.cc:2369] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:08:36.821353 24480 raft_consensus.cc:482] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:08:36.821520 24480 raft_consensus.cc:504] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.821944 24480 leader_election.cc:288] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcdc46b41704e28bff5f43da6cf9cc1 (127.3.236.131:45853), bbce6983db0e4433afb34936adf6e67e (127.3.236.129:38881)
I20260811 07:08:36.826362 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.822276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:41317 (local address 127.3.236.131:45853)
0811 07:08:36.822324 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.822327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.822445 (+ 118us) server_negotiation.cc:407] Connection header received
0811 07:08:36.822566 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.822567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.822580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.822614 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:08:36.822855 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.823666 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.825373 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.825953 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.826009 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.826016 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.826248 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.826289 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:08:36.826362 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.822610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:42029 (local address 127.3.236.129:38881)
0811 07:08:36.822666 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.822668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.822680 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:08:36.822711 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.822712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.822721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.822801 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:08:36.823042 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.823728 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.825374 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.825952 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.826005 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.826013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.826248 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.826289 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:08:36.826629 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.822425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38881 (local address 127.3.236.130:42029)
0811 07:08:36.822597 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.822607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.822653 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.822818 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.822818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.822820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:08:36.822950 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.822952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.823816 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.823816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.825283 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.825285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.826080 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.826081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.826160 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.826165 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.826456 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.826527 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":52,"thread_start_us":50,"threads_started":1}
I20260811 07:08:36.826927 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.822221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:45853 (local address 127.3.236.130:41317)
0811 07:08:36.822378 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.822389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:08:36.822476 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:08:36.822694 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:08:36.822695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:08:36.822697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:08:36.822769 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.822773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.823808 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.823810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.825211 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:08:36.825215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.826088 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.826088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:08:36.826160 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.826164 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:08:36.826804 (+ 640us) client_negotiation.cc:240] Negotiation successful
0811 07:08:36.826854 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":100,"thread_start_us":83,"threads_started":1}
I20260811 07:08:36.827014 23842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e766c343224dafbf887b2162d3b0b5" candidate_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbce6983db0e4433afb34936adf6e67e" is_pre_election: true
I20260811 07:08:36.827193 23842 raft_consensus.cc:2454] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb5a95f936e048f78c9d3f944b7f3bce in term 0.
I20260811 07:08:36.827406 23965 leader_election.cc:302] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb5a95f936e048f78c9d3f944b7f3bce, bbce6983db0e4433afb34936adf6e67e; no voters:
I20260811 07:08:36.827581 24480 raft_consensus.cc:2781] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:08:36.827648 24480 raft_consensus.cc:482] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:08:36.827678 24480 raft_consensus.cc:3037] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:08:36.828447 24480 raft_consensus.cc:504] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.828583 24480 leader_election.cc:288] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [CANDIDATE]: Term 1 election: Requested vote from peers 1dcdc46b41704e28bff5f43da6cf9cc1 (127.3.236.131:45853), bbce6983db0e4433afb34936adf6e67e (127.3.236.129:38881)
I20260811 07:08:36.828751 23842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e766c343224dafbf887b2162d3b0b5" candidate_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbce6983db0e4433afb34936adf6e67e"
I20260811 07:08:36.828752 24346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e766c343224dafbf887b2162d3b0b5" candidate_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" is_pre_election: true
I20260811 07:08:36.828835 23842 raft_consensus.cc:3037] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:08:36.828877 24346 raft_consensus.cc:2454] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb5a95f936e048f78c9d3f944b7f3bce in term 0.
I20260811 07:08:36.829006 24345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e766c343224dafbf887b2162d3b0b5" candidate_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1"
I20260811 07:08:36.829072 24345 raft_consensus.cc:3037] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:08:36.829722 24345 raft_consensus.cc:2454] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb5a95f936e048f78c9d3f944b7f3bce in term 1.
I20260811 07:08:36.829726 23842 raft_consensus.cc:2454] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb5a95f936e048f78c9d3f944b7f3bce in term 1.
I20260811 07:08:36.829900 23965 leader_election.cc:302] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcdc46b41704e28bff5f43da6cf9cc1, bb5a95f936e048f78c9d3f944b7f3bce; no voters:
I20260811 07:08:36.830005 24480 raft_consensus.cc:2781] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:08:36.830188 24480 raft_consensus.cc:686] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 1 LEADER]: Becoming Leader. State: Replica: bb5a95f936e048f78c9d3f944b7f3bce, State: Running, Role: LEADER
I20260811 07:08:36.830312 24480 consensus_queue.cc:237] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } }
I20260811 07:08:36.831220 23542 catalog_manager.cc:5495] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce reported cstate change: term changed from 0 to 1, leader changed from <none> to bb5a95f936e048f78c9d3f944b7f3bce (127.3.236.130). New cstate: current_term: 1 leader_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5a95f936e048f78c9d3f944b7f3bce" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 } health_report { overall_health: UNKNOWN } } }
I20260811 07:08:36.851620 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.849077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.3.236.130:33665)
0811 07:08:36.849142 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:08:36.849144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:08:36.849151 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:08:36.849196 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:08:36.849197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:08:36.849215 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:08:36.849292 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:08:36.849481 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.849947 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.850739 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:08:36.850953 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:08:36.851049 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:08:36.851229 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:08:36.851342 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:08:36.851435 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:08:36.851478 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 07:08:36.851498 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:08:36.853745 24345 raft_consensus.cc:1261] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 1 FOLLOWER]: Refusing update from remote peer bb5a95f936e048f78c9d3f944b7f3bce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:08:36.853747 23842 raft_consensus.cc:1261] T f9e766c343224dafbf887b2162d3b0b5 P bbce6983db0e4433afb34936adf6e67e [term 1 FOLLOWER]: Refusing update from remote peer bb5a95f936e048f78c9d3f944b7f3bce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:08:36.854208 24488 consensus_queue.cc:1035] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbce6983db0e4433afb34936adf6e67e" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:08:36.854458 24480 consensus_queue.cc:1035] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcdc46b41704e28bff5f43da6cf9cc1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:08:36.857440 24491 mvcc.cc:205] Tried to move back new op lower bound from 7317225950629662720 to 7317225950537547776. Current Snapshot: MvccSnapshot[applied={T|T < 7317225950629662720}]
I20260811 07:08:36.858762 24493 mvcc.cc:205] Tried to move back new op lower bound from 7317225950629662720 to 7317225950537547776. Current Snapshot: MvccSnapshot[applied={T|T < 7317225950629662720}]
I20260811 07:08:36.859863 23541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:08:36.859959 23541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:08:36.860772 23541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe95654e0721482fa695ba64370b53a6: Sending DeleteTablet for 3 replicas of tablet f9e766c343224dafbf887b2162d3b0b5
I20260811 07:08:36.861178 24060 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e766c343224dafbf887b2162d3b0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:08:36 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:43352
I20260811 07:08:36.861585 24311 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e766c343224dafbf887b2162d3b0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:08:36 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:39634
I20260811 07:08:36.861673 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 23684
I20260811 07:08:36.861802 23799 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e766c343224dafbf887b2162d3b0b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:08:36 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34680
I20260811 07:08:36.874760 24511 tablet_replica.cc:330] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: stopping tablet replica
I20260811 07:08:36.874954 24511 raft_consensus.cc:2229] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:08:36.875150 24511 raft_consensus.cc:2258] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:08:36.878450 24511 ts_tablet_manager.cc:1905] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:08:36.885633 24511 ts_tablet_manager.cc:1918] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:08:36.885743 24511 log.cc:1196] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/f9e766c343224dafbf887b2162d3b0b5
I20260811 07:08:36.886049 24511 ts_tablet_manager.cc:1939] T f9e766c343224dafbf887b2162d3b0b5 P 1dcdc46b41704e28bff5f43da6cf9cc1: Deleting consensus metadata
I20260811 07:08:36.886530 23516 catalog_manager.cc:4841] TS 1dcdc46b41704e28bff5f43da6cf9cc1 (127.3.236.131:45853): tablet f9e766c343224dafbf887b2162d3b0b5 (table test-table [id=313c30a6377f4ea880ef582bf5e74f2a]) successfully deleted
W20260811 07:08:36.887696 23965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:38881 (error 108)
W20260811 07:08:36.888119 23516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:38881 (error 108)
I20260811 07:08:36.888370 24510 tablet_replica.cc:330] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce: stopping tablet replica
I20260811 07:08:36.888500 24510 raft_consensus.cc:2229] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:08:36.888670 24510 raft_consensus.cc:2258] T f9e766c343224dafbf887b2162d3b0b5 P bb5a95f936e048f78c9d3f944b7f3bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:08:36.889070 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 23929
I20260811 07:08:36.924038 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.923827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38881 (local address 127.0.0.1:34686)
0811 07:08:36.923914 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.924002 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 07:08:36.925590 23516 catalog_manager.cc:4584] TS bbce6983db0e4433afb34936adf6e67e (127.3.236.129:38881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9e766c343224dafbf887b2162d3b0b5: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38881: connect: Connection refused (error 111)
I20260811 07:08:36.930963 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:08:36.930830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:33665 (local address 127.0.0.1:43376)
0811 07:08:36.930885 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:08:36.930908 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.130:33665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:08:36.931178 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 24192
I20260811 07:08:36.943464 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 23483
I20260811 07:08:36.964459 4018 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 07:08:37.321403 4018 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 07:08:37.321535 4018 mini_ranger.cc:66] Stopping Ranger...
I20260811 07:08:37.675700 4018 mini_ranger.cc:68] Stopped Ranger
2026-08-11 07:08:37.675 UTC [22724] LOG: received smart shutdown request
2026-08-11 07:08:37.680 UTC [22724] LOG: background worker "logical replication launcher" (PID 22742) exited with exit code 1
2026-08-11 07:08:37.680 UTC [22732] LOG: shutting down
2026-08-11 07:08:37.852 UTC [22724] LOG: database system is shut down
2026-08-11T07:08:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49942 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:08:38 dist-test-slave-2d19 krb5kdc[24535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:08:38 dist-test-slave-2d19 krb5kdc[24535](info): set up 2 sockets
Aug 11 07:08:38 dist-test-slave-2d19 krb5kdc[24535](info): commencing operation
krb5kdc: starting...
W20260811 07:08:39.259768 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:08:39 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:08:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangeradmin_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangeradmin_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangerlookup_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangerlookup_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
I20260811 07:08:39.434996 4018 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhrK1s3/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/postgres -l logfile start
2026-08-11 07:08:41.967 UTC [24593] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 07:08:41.967 UTC [24593] LOG: listening on IPv4 address "127.3.236.148", port 46213
2026-08-11 07:08:41.971 UTC [24593] LOG: listening on Unix socket "/tmp/.s.PGSQL.46213"
2026-08-11 07:08:42.007 UTC [24597] LOG: database system was shut down at 2026-08-11 07:08:40 UTC
2026-08-11 07:08:42.012 UTC [24593] LOG: database system is ready to accept connections
I20260811 07:08:43.220556 4018 mini_postgres.cc:96] Postgres bound to 46213
2026-08-11 07:08:43.227 UTC [24610] FATAL: database "slave" does not exist
127.3.236.148:46213 - accepting connections
I20260811 07:08:43.228493 4018 mini_ranger.cc:161] Starting Ranger...
I20260811 07:08:43.246193 4018 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 07:08:43.739300 4018 mini_ranger.cc:87] Created ranger Postgres database
I20260811 07:08:43.739372 4018 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin
2026-08-11 07:08:44,034 [I] DB FLAVOR :POSTGRES
2026-08-11 07:08:44,035 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:08:44,035 [I] Checking connection..
2026-08-11 07:08:44,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 07:08:44,300 [I] Checking connection passed.
2026-08-11 07:08:44,300 [I] --------- Verifying version history table ---------
2026-08-11 07:08:44,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:08:44,575 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:08:44,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:08:44,817 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:08:44,817 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 07:08:44,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 07:08:45.050 UTC [24727] WARNING: there is no transaction in progress
2026-08-11 07:08:45,060 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 07:08:45,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:08:45,326 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 07:08:45,326 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:08:45,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 07:08:45,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 07:08:45,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2d19.c.gcp-upstream.internal','N') ;"
2026-08-11 07:08:46,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:08:46,344 [I] Table x_portal_user does not exist in database ranger
2026-08-11 07:08:46,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:08:46,595 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 07:08:46,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:08:46,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:08:47,119 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 07:08:47,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 07:08:48.097 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.114 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.130 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.146 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.165 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.186 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.251 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.259 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.274 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.286 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.297 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.311 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.321 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.327 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.334 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:48.340 UTC [24910] WARNING: there is no transaction in progress
2026-08-11 07:08:49,022 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 07:08:49,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 07:08:49,285 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 07:08:49,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:08:49,531 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 07:08:49,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:08:49,778 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 07:08:49,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:08:50,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:08:50,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2d19.c.gcp-upstream.internal';"
2026-08-11 07:08:50,567 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 07:08:50,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 07:08:50,835 [I] DB_PATCHES have already been applied
I20260811 07:08:50.841776 4018 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhrK1s3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 07:08:50.841842 4018 mini_ranger.cc:193] Using host: 127.3.236.148
I20260811 07:08:50.845777 4018 mini_ranger.cc:227] Ranger admin URL: http://127.3.236.148:47025
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:08:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 07:08:52.276219 4018 mini_ranger.cc:160] Time spent starting Ranger: real 9.048s user 0.001s sys 0.010s
Aug 11, 2026 7:09:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:09:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 7:09:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:09:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:09:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 07:09:13.697739 4018 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangerkms_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/rangerkms_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/HTTP_127.3.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 07:09:13 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432153, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 07:09:13.787254 4018 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 07:09:13.797640 4018 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 07:09:14.722035 4018 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 07:09:14.722126 4018 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms
I20260811 07:09:14.722144 4018 mini_ranger_kms.cc:227] Using postgres at 127.3.236.148:46213
2026-08-11 07:09:14,930 [I] DB FLAVOR :POSTGRES
2026-08-11 07:09:14,931 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:09:14,931 [I] Checking connection
2026-08-11 07:09:14,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 07:09:15,204 [I] connection success
2026-08-11 07:09:15,204 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 07:09:15,204 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 07:09:15,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 07:09:15,445 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 07:09:15,445 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:09:15,445 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 07:09:15,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.236.148:46213/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 07:09:15,740 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 07:09:16.225502 4018 mini_ranger_kms.cc:310] Created kms service
I20260811 07:09:16.441040 4018 mini_ranger_kms.cc:326] Created kudu user
I20260811 07:09:16.492213 4018 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 07:09:16 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432153, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:16.947091 4018 mini_ranger_kms.cc:384] Added ranger policy
I20260811 07:09:16.947217 4018 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhrK1s3/build/release/bin/postgresql.jar:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhrK1s3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 07:09:16.947237 4018 mini_ranger_kms.cc:240] Using host: 127.3.236.148
I20260811 07:09:16.951745 4018 mini_ranger_kms.cc:276] Ranger KMS PID: 25238
I20260811 07:09:16.951849 4018 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.236.148:57425
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 07:09:19 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786432159, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:19.880875 4018 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.094s user 0.003s sys 0.189s
I20260811 07:09:19.881000 4018 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 07:09:19.881047 4018 mini_ranger_kms.cc:401] 127.3.236.148:57425/kms/v1/keys
Aug 11 07:09:20 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432159, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
Aug 11 07:09:20 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432159, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.236.148@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
Aug 11 07:09:22 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:09:22.719182 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38069
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:38069
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:22.804354 25327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:22.804572 25327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:22.804595 25327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:22.805765 25327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:22.805811 25327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:22.805825 25327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:22.805838 25327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:22.805850 25327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:22.807432 25327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.236.148:57425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:38069
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:22.807684 25327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:22.807926 25327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:22.813416 25333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:22.813421 25336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:22.813408 25343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:22.813582 25327 server_base.cc:1034] running on GCE node
I20260811 07:09:22.813913 25327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:22.814221 25327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:22.815398 25327 hybrid_clock.cc:648] HybridClock initialized: now 1786432162815372 us; error 42 us; skew 500 ppm
Aug 11 07:09:22 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432162, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:22.823033 25327 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:22.823575 25327 webserver.cc:460] Webserver started at http://127.3.236.190:33105/ using document root <none> and password file <none>
I20260811 07:09:22.823823 25327 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:22.823884 25327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:22.824048 25327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:09:22 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432162, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:22.892565 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "7d3630a5111f40fe8247a365e071fcb5"
format_stamp: "Formatted at 2026-08-11 07:09:22 on dist-test-slave-2d19"
server_key: "3f0e2372994fcc0567fd7ec230b28d75"
server_key_iv: "24c9f6eeeb275c1bcf9b4d1f3f431022"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:22.893111 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "7d3630a5111f40fe8247a365e071fcb5"
format_stamp: "Formatted at 2026-08-11 07:09:22 on dist-test-slave-2d19"
server_key: "3f0e2372994fcc0567fd7ec230b28d75"
server_key_iv: "24c9f6eeeb275c1bcf9b4d1f3f431022"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:22.895192 25327 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:09:22.912804 25361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:22.913100 25327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:09:22.913233 25327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "7d3630a5111f40fe8247a365e071fcb5"
format_stamp: "Formatted at 2026-08-11 07:09:22 on dist-test-slave-2d19"
server_key: "3f0e2372994fcc0567fd7ec230b28d75"
server_key_iv: "24c9f6eeeb275c1bcf9b4d1f3f431022"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:22.913354 25327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:22.936864 25327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:22.939299 25327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:22.939515 25327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:22.945114 25327 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:38069
I20260811 07:09:22.945192 25463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:38069 every 8 connection(s)
I20260811 07:09:22.945617 25327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:22.946476 25466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:22.947860 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 25327
I20260811 07:09:22.947961 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:09:22.948186 4018 external_mini_cluster.cc:1428] Setting key 15240958b365e62f4dd754e81a98a75f
I20260811 07:09:22.949410 25466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Bootstrap starting.
I20260811 07:09:22.950089 25466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:22.950421 25466 log.cc:824] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:22.951298 25466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: No bootstrap required, opened a new log
I20260811 07:09:22.953730 25466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:22.953912 25466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:22.954000 25466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3630a5111f40fe8247a365e071fcb5, State: Initialized, Role: FOLLOWER
I20260811 07:09:22.954164 25466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:22.954262 25466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:22.954311 25466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:22.954365 25466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:22.954959 25466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:22.955093 25466 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3630a5111f40fe8247a365e071fcb5; no voters:
Aug 11 07:09:22 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:22.955292 25466 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:09:22.955355 25475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:22.955581 25475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3630a5111f40fe8247a365e071fcb5, State: Running, Role: LEADER
I20260811 07:09:22.955722 25475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:22.955600 25466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:22.956084 25475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3630a5111f40fe8247a365e071fcb5. Latest consensus state: current_term: 1 leader_uuid: "7d3630a5111f40fe8247a365e071fcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } } }
I20260811 07:09:22.956279 25475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:22.956324 25476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d3630a5111f40fe8247a365e071fcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } } }
I20260811 07:09:22.956422 25476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:22.956918 25492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:22.957688 25492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:22.958045 25466 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 07:09:22.963908 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:22.948820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.3.236.190:38069)
0811 07:09:22.949049 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:22.949057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:22.949082 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:09:22.949486 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:22.949494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:22.949575 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:22.949800 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:22.950486 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:22.951644 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:22.952499 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:22.952904 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:22.953029 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:22.955897 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:22.955902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:22.955902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:22.955915 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:22.958381 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:22.958754 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:22.958757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:22.958757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:22.960849 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:22.961239 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:22.961240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:22.961241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:22.962050 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:22.962847 (+ 797us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:22.962949 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 07:09:22.963531 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260811 07:09:22.964128 25492 catalog_manager.cc:1347] Generated new cluster ID: 6029f1c7147346ef8fdf998e85e8c8f9
I20260811 07:09:22.964237 25492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:22.989717 25492 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:09:22.990346 25492 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:22.995164 25492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Generated new TSK 0
I20260811 07:09:22.995368 25492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:09:23.703907 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:23.814291 25566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:23.814579 25566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:23.814620 25566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:23.816422 25566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:23.816480 25566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:23.816521 25566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:23.818682 25566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:23.818928 25566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:23.819247 25566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:23.826280 25576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:23.826370 25577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:23.826450 25580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:23.827064 25566 server_base.cc:1034] running on GCE node
I20260811 07:09:23.827286 25566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:23.827553 25566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:23.828709 25566 hybrid_clock.cc:648] HybridClock initialized: now 1786432163828682 us; error 42 us; skew 500 ppm
Aug 11 07:09:23 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432163, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:23.831651 25566 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:23.832293 25566 webserver.cc:460] Webserver started at http://127.3.236.129:41487/ using document root <none> and password file <none>
I20260811 07:09:23.832635 25566 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:23.832710 25566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:23.832860 25566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:09:23 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432163, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:23.846552 25566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "1de8d747a6234581b061b13638e9d465"
format_stamp: "Formatted at 2026-08-11 07:09:23 on dist-test-slave-2d19"
server_key: "ac7463022e9c1f31d8e2e6b76f133a50"
server_key_iv: "db631f96d2877c473d4a5d20dbe7598e"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:23.846963 25566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "1de8d747a6234581b061b13638e9d465"
format_stamp: "Formatted at 2026-08-11 07:09:23 on dist-test-slave-2d19"
server_key: "ac7463022e9c1f31d8e2e6b76f133a50"
server_key_iv: "db631f96d2877c473d4a5d20dbe7598e"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:23.848641 25566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:09:23.862247 25592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:23.862553 25566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:23.862666 25566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "1de8d747a6234581b061b13638e9d465"
format_stamp: "Formatted at 2026-08-11 07:09:23 on dist-test-slave-2d19"
server_key: "ac7463022e9c1f31d8e2e6b76f133a50"
server_key_iv: "db631f96d2877c473d4a5d20dbe7598e"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:23.862766 25566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:23.889981 25566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:23.892462 25566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:23.892642 25566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:23.892915 25566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:23.893311 25566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:23.893363 25566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:23.893433 25566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:23.893463 25566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:23.902407 25566 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:33067
I20260811 07:09:23.902509 25786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:33067 every 8 connection(s)
I20260811 07:09:23.902859 25566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:09:23 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432163, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:23.911782 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 25566
I20260811 07:09:23.911907 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:09:23.912142 4018 external_mini_cluster.cc:1428] Setting key 865e492804b6351bf2c8cc9d4539107a
I20260811 07:09:23.915640 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:23.903637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.129:50249)
0811 07:09:23.903895 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:23.903951 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:23.904816 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:23.905193 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:23.905198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:23.905462 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:23.905780 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:23.905787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:23.906524 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:23.906525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:23.907891 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:23.907894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:23.908508 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:23.908510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:23.908649 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:23.909361 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:23.909371 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:23.911264 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:23.913709 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:23.913718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:23.913721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:23.914076 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:23.914334 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:23.914335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:23.914335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:23.914506 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:23.914848 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:23.914854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:23.915047 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 07:09:23.915173 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 07:09:23.915355 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":114,"threads_started":1}
I20260811 07:09:23.916141 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:23.903733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:50249 (local address 127.3.236.190:38069)
0811 07:09:23.903928 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:23.903930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:23.904072 (+ 142us) server_negotiation.cc:407] Connection header received
0811 07:09:23.904930 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:23.904931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:23.904942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:23.905090 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:23.905880 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:23.906426 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:23.907998 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:23.908389 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:23.908488 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:23.911370 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:23.911371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:23.911371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:23.911378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:23.913583 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:23.914178 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:23.914179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:23.914180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:23.914253 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:23.914595 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:23.914596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:23.914596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:23.914751 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:23.914817 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:23.915925 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0811 07:09:23.916031 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":127,"threads_started":1}
I20260811 07:09:23.916566 25787 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:23.916678 25787 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:23.916906 25787 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:23.917521 25392 ts_manager.cc:194] Registered new tserver with Master: 1de8d747a6234581b061b13638e9d465 (127.3.236.129:33067)
I20260811 07:09:23.918524 25392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:50249
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:09:23.948271 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:24.060731 25797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:24.060982 25797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:24.061020 25797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:24.062292 25797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:24.062367 25797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:24.062415 25797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:24.064056 25797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:24.064373 25797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:24.064646 25797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:24.070621 25809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.070597 25811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.071085 25807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.071656 25797 server_base.cc:1034] running on GCE node
I20260811 07:09:24.071870 25797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:24.072168 25797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:24.073324 25797 hybrid_clock.cc:648] HybridClock initialized: now 1786432164073310 us; error 49 us; skew 500 ppm
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:24.076485 25797 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:24.076936 25797 webserver.cc:460] Webserver started at http://127.3.236.130:44467/ using document root <none> and password file <none>
I20260811 07:09:24.077198 25797 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:24.077267 25797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:24.077409 25797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:24.088330 25797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "12a0e4195469440f967b81e84b0db16a"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "4cb0733063682423d430a5dc0235143f"
server_key_iv: "b40dbc0babec8e46e202acc011cb8f63"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.088793 25797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "12a0e4195469440f967b81e84b0db16a"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "4cb0733063682423d430a5dc0235143f"
server_key_iv: "b40dbc0babec8e46e202acc011cb8f63"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.090775 25797 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:24.104389 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.104626 25797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:24.104723 25797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "12a0e4195469440f967b81e84b0db16a"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "4cb0733063682423d430a5dc0235143f"
server_key_iv: "b40dbc0babec8e46e202acc011cb8f63"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.104808 25797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:24.135965 25797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:24.138526 25797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:24.138777 25797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:24.139081 25797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:24.139544 25797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:24.139642 25797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.139731 25797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:24.139803 25797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.147163 25797 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:40671
I20260811 07:09:24.147511 26028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:40671 every 8 connection(s)
I20260811 07:09:24.147653 25797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:09:24.153146 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 25797
I20260811 07:09:24.153244 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:09:24.153446 4018 external_mini_cluster.cc:1428] Setting key 669a591a49420e09fe1a8ff6281f3e15
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:24.166857 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.148903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:43171 (local address 127.3.236.190:38069)
0811 07:09:24.148959 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:24.148961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:24.150740 (+ 1779us) server_negotiation.cc:407] Connection header received
0811 07:09:24.151296 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:24.151298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:24.151313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:24.151372 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:24.152051 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.152809 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.158534 (+ 5725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.158889 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.158963 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.161861 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:24.161863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:24.161863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:24.161873 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:24.163876 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.165421 (+ 1545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.165423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.165423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.165510 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.165924 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.165925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.165926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.166079 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:24.166140 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:24.166568 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 07:09:24.166695 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 07:09:24.167515 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.148567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.130:43171)
0811 07:09:24.150567 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:24.150622 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:24.151180 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:24.151515 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:24.151520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:24.151729 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:24.151938 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.151942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.156796 (+ 4854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.156798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.158419 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.158422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.159054 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.159055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.159222 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.159891 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:24.159902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:24.161744 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:24.164861 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.164870 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.164873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.165308 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.165617 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.165619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.165619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.165811 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.166180 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:24.166187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:24.166373 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 07:09:24.167019 (+ 646us) client_negotiation.cc:240] Negotiation successful
0811 07:09:24.167193 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1945,"thread_start_us":99,"threads_started":1}
I20260811 07:09:24.168238 26029 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:24.168386 26029 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:24.168598 26029 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
I20260811 07:09:24.169031 25392 ts_manager.cc:194] Registered new tserver with Master: 12a0e4195469440f967b81e84b0db16a (127.3.236.130:40671)
I20260811 07:09:24.169445 25392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:43171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:09:24.191373 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:24.325035 26039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:24.325384 26039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:24.325490 26039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:24.327222 26039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:24.327327 26039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:24.327409 26039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:24.329638 26039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:24.330083 26039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:24.330411 26039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:24.337368 26048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.337728 26046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.337812 26039 server_base.cc:1034] running on GCE node
W20260811 07:09:24.337951 26045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.338383 26039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:24.338697 26039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:24.339877 26039 hybrid_clock.cc:648] HybridClock initialized: now 1786432164339854 us; error 37 us; skew 500 ppm
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:24.343778 26039 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:24.344399 26039 webserver.cc:460] Webserver started at http://127.3.236.131:41889/ using document root <none> and password file <none>
I20260811 07:09:24.344738 26039 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:24.344807 26039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:24.345007 26039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:24.367569 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "f7cd75be07cb4da5b0a1b57171f79850"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "a1c8748c8d46a2b42c70c3263d449c0a"
server_key_iv: "f378869eaf9cd926b0df8032eba2e717"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.368196 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "f7cd75be07cb4da5b0a1b57171f79850"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "a1c8748c8d46a2b42c70c3263d449c0a"
server_key_iv: "f378869eaf9cd926b0df8032eba2e717"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.372047 26039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260811 07:09:24.385936 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.386214 26039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:24.386386 26039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f7cd75be07cb4da5b0a1b57171f79850"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "a1c8748c8d46a2b42c70c3263d449c0a"
server_key_iv: "f378869eaf9cd926b0df8032eba2e717"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.386574 26039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:24.412457 26039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:24.414975 26039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:24.415143 26039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:24.415436 26039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:24.415886 26039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:24.415938 26039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.415971 26039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:24.416041 26039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.424826 26039 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:41105
I20260811 07:09:24.424937 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:41105 every 8 connection(s)
I20260811 07:09:24.425349 26039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:09:24.430457 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 26039
I20260811 07:09:24.430564 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:09:24.430758 4018 external_mini_cluster.cc:1428] Setting key 8be25ea6a76c889e065ae90c176eb620
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:24.438752 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.426127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:43729 (local address 127.3.236.190:38069)
0811 07:09:24.426181 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:24.426183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:24.426436 (+ 253us) server_negotiation.cc:407] Connection header received
0811 07:09:24.427149 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:24.427150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:24.427163 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:24.427195 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:24.428046 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.428922 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.431159 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.431588 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.431640 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.434895 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:24.434896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:24.434897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:24.434906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:24.437019 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.437560 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.437562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.437562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.437655 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.437963 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.437964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.437964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.438124 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:24.438207 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:24.438565 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 07:09:24.438648 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:09:24.438932 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.426112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.131:43729)
0811 07:09:24.426283 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:24.426332 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:24.427020 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:24.427385 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:24.427389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:24.427649 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:24.427929 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.427936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.429042 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.429043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.431042 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.431046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.431762 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.431764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.431992 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.432719 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:24.432731 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:24.434788 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:24.437137 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.437145 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.437147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.437475 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.437740 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.437741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.437741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.437885 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.438208 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:24.438215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:24.438396 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 07:09:24.438529 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 07:09:24.438692 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260811 07:09:24.452821 26300 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:24.452973 26300 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:24.453202 26300 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:24.453714 25392 ts_manager.cc:194] Registered new tserver with Master: f7cd75be07cb4da5b0a1b57171f79850 (127.3.236.131:41105)
I20260811 07:09:24.454272 25392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:43729
I20260811 07:09:24.459741 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:09:24.460700 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 25566
I20260811 07:09:24.489609 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 25797
I20260811 07:09:24.500408 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 26039
I20260811 07:09:24.514273 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 25327
I20260811 07:09:24.527082 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38069
--webserver_interface=127.3.236.190
--webserver_port=33105
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:38069
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:24.611337 26307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:24.611577 26307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:24.611614 26307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:24.612818 26307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:24.612879 26307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:24.612905 26307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:24.612934 26307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:24.612972 26307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:24.614516 26307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:38069
--ranger_config_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=33105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:24.614792 26307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:24.615011 26307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:24.619802 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.619797 26323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.619807 26319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.620190 26307 server_base.cc:1034] running on GCE node
I20260811 07:09:24.620465 26307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:24.620772 26307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:24.621914 26307 hybrid_clock.cc:648] HybridClock initialized: now 1786432164621807 us; error 114 us; skew 500 ppm
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:24.624394 26307 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:24.624819 26307 webserver.cc:460] Webserver started at http://127.3.236.190:33105/ using document root <none> and password file <none>
I20260811 07:09:24.625056 26307 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:24.625126 26307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:24.704566 26307 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:24.705361 26333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.705602 26307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:24.705721 26307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "7d3630a5111f40fe8247a365e071fcb5"
format_stamp: "Formatted at 2026-08-11 07:09:22 on dist-test-slave-2d19"
server_key: "3f0e2372994fcc0567fd7ec230b28d75"
server_key_iv: "24c9f6eeeb275c1bcf9b4d1f3f431022"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.706101 26307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:24.720713 26307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:24.722368 26307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:24.722524 26307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:24.726990 26307 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:38069
I20260811 07:09:24.727051 26445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:38069 every 8 connection(s)
I20260811 07:09:24.727396 26307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:24.728729 26446 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:09:24.729377 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Bootstrap starting.
I20260811 07:09:24.733856 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 26307
I20260811 07:09:24.734189 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:33067
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:38069
--webserver_port=41487
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:24.737767 26446 log.cc:824] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:24.739594 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:09:24.739888 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Bootstrap complete.
I20260811 07:09:24.742409 26446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:24.742684 26446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3630a5111f40fe8247a365e071fcb5, State: Initialized, Role: FOLLOWER
I20260811 07:09:24.742910 26446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:24.743010 26446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:24.743078 26446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:24.743134 26446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:09:24.744184 26446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:24.744316 26446 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3630a5111f40fe8247a365e071fcb5; no voters:
I20260811 07:09:24.744511 26446 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:09:24.744580 26451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:09:24.744822 26446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:24.744846 26451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [term 2 LEADER]: Becoming Leader. State: Replica: 7d3630a5111f40fe8247a365e071fcb5, State: Running, Role: LEADER
I20260811 07:09:24.745122 26451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } }
I20260811 07:09:24.745463 26453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d3630a5111f40fe8247a365e071fcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } } }
I20260811 07:09:24.745484 26455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3630a5111f40fe8247a365e071fcb5. Latest consensus state: current_term: 2 leader_uuid: "7d3630a5111f40fe8247a365e071fcb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3630a5111f40fe8247a365e071fcb5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38069 } } }
I20260811 07:09:24.745546 26453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:24.745577 26455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:24.745893 26468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:24.746403 26468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:24.746587 26468 catalog_manager.cc:1259] Loaded cluster ID: 6029f1c7147346ef8fdf998e85e8c8f9
I20260811 07:09:24.746662 26468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:24.746770 26446 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 07:09:24.750934 26468 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:24.751577 26468 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Loaded TSK: 0
I20260811 07:09:24.751894 26468 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:09:24.852303 26449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:24.852530 26449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:24.852577 26449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:24.853765 26449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:24.853832 26449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:24.853876 26449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:24.855973 26449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:33067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=41487
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:24.856357 26449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:24.856686 26449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:24.863438 26515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:24.863488 26523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.864626 26449 server_base.cc:1034] running on GCE node
W20260811 07:09:24.864842 26510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:24.865129 26449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:24.865422 26449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:24.866582 26449 hybrid_clock.cc:648] HybridClock initialized: now 1786432164866565 us; error 34 us; skew 500 ppm
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:24.869376 26449 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:24.869925 26449 webserver.cc:460] Webserver started at http://127.3.236.129:41487/ using document root <none> and password file <none>
I20260811 07:09:24.870175 26449 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:24.870258 26449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:24.894054 26449 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:24.900005 26541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.900394 26449 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260811 07:09:24.900564 26449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "1de8d747a6234581b061b13638e9d465"
format_stamp: "Formatted at 2026-08-11 07:09:23 on dist-test-slave-2d19"
server_key: "ac7463022e9c1f31d8e2e6b76f133a50"
server_key_iv: "db631f96d2877c473d4a5d20dbe7598e"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:24.901161 26449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:24.926074 26449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:24.928501 26449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:24.928689 26449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:24.928982 26449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:24.929370 26449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:24.929423 26449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.929467 26449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:24.929493 26449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:24.937100 26449 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:33067
I20260811 07:09:24.937160 26747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:33067 every 8 connection(s)
I20260811 07:09:24.937587 26449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:09:24.944890 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 26449
I20260811 07:09:24.945286 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:40671
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:38069
--webserver_port=44467
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:09:24 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432164, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:24.950963 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.938408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.129:42239)
0811 07:09:24.938603 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:24.938653 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:24.939228 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:24.939750 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:24.939755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:24.940006 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:24.940321 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.940329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.942055 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.942056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.943261 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:24.943264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.944090 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.944092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:24.944231 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.944907 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:24.944921 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:24.946511 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:24.948926 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.948933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.948936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.949265 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.949586 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:24.949587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:24.949587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:24.949731 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:24.950265 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:24.950272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:24.950433 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 07:09:24.950544 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 07:09:24.950710 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260811 07:09:24.951845 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:24.938493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:42239 (local address 127.3.236.190:38069)
0811 07:09:24.938744 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:24.938767 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:24.938815 (+ 48us) server_negotiation.cc:407] Connection header received
0811 07:09:24.939471 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:24.939482 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:24.939527 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:24.939668 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:24.940473 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.941903 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.943372 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:24.943824 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:24.943955 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:24.946648 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:24.946652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:24.946652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:24.946663 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:24.948841 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.949370 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.949374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.949374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.949499 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:24.949812 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:24.949813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:24.949813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:24.950145 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:24.950233 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:24.951497 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0811 07:09:24.951643 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260811 07:09:24.952613 26749 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:24.952785 26749 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:24.953059 26749 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:24.954209 26749 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:38069 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:09:24.954367 26749 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:24.954510 26749 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:24.955016 26749 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:24.955152 26749 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:24.955582 26749 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:09:25.092123 26758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:25.092453 26758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:25.092496 26758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:25.094398 26758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:25.094471 26758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:25.094517 26758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:25.097117 26758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:40671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=44467
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:25.097558 26758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:25.097950 26758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:25.104108 26770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:25.104141 26766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:25.104508 26758 server_base.cc:1034] running on GCE node
W20260811 07:09:25.104307 26767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:25.104803 26758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:25.105141 26758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:25.106345 26758 hybrid_clock.cc:648] HybridClock initialized: now 1786432165106330 us; error 32 us; skew 500 ppm
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:25.110028 26758 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:25.110672 26758 webserver.cc:460] Webserver started at http://127.3.236.130:44467/ using document root <none> and password file <none>
I20260811 07:09:25.110981 26758 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:25.111088 26758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:25.138746 26758 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260811 07:09:25.140084 26781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.140307 26758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:25.140427 26758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "12a0e4195469440f967b81e84b0db16a"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "4cb0733063682423d430a5dc0235143f"
server_key_iv: "b40dbc0babec8e46e202acc011cb8f63"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:25.140992 26758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:25.176699 26758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:25.179281 26758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:25.179541 26758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:25.179931 26758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:25.180533 26758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:25.180616 26758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.180670 26758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:25.180691 26758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.188839 26758 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:40671
I20260811 07:09:25.188923 26981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:40671 every 8 connection(s)
I20260811 07:09:25.189355 26758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:09:25.195623 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 26758
I20260811 07:09:25.195971 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.236.148:57425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:41105
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:38069
--webserver_port=41889
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:25.200979 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:25.190223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.130:51379)
0811 07:09:25.190419 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:25.190465 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:25.190999 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:25.191283 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:25.191287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:25.191484 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:25.191697 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:25.191701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.192728 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.192730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:25.193856 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:25.193859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.194369 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.194370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:25.194476 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:25.195073 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:25.195083 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:25.196975 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:25.199052 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:25.199061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:25.199063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:25.199439 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:25.199719 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:25.199721 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:25.199721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:25.199882 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:25.200218 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:25.200225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:25.200421 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 07:09:25.200561 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 07:09:25.200735 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260811 07:09:25.201736 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:25.190241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:51379 (local address 127.3.236.190:38069)
0811 07:09:25.190320 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:25.190324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:25.190582 (+ 258us) server_negotiation.cc:407] Connection header received
0811 07:09:25.191113 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:25.191115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:25.191132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:25.191227 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:25.191794 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.192627 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.193956 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.194259 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.194346 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:25.197085 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:25.197086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:25.197087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:25.197095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:25.198934 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.199532 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.199533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.199534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.199614 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.199969 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.199970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.199971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.200113 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:25.200185 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:25.201534 (+ 1349us) server_negotiation.cc:299] Negotiation successful
0811 07:09:25.201637 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:09:25.202299 26984 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:25.202423 26984 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.202641 26984 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:25.203347 26984 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:38069 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:09:25.203425 26984 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.203491 26984 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:25.203879 26984 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.203961 26984 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:25.204427 26984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:09:25.337208 26987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:25.337424 26987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:25.337451 26987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:25.338953 26987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:25.339006 26987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:25.339036 26987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:25.341369 26987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.236.148:57425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:41105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=41889
--tserver_master_addrs=127.3.236.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:25.341667 26987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:25.341982 26987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:25.349449 27008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:25.349466 27005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:25.350317 27011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:25.350698 26987 server_base.cc:1034] running on GCE node
I20260811 07:09:25.350987 26987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:25.351279 26987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:25.352458 26987 hybrid_clock.cc:648] HybridClock initialized: now 1786432165352446 us; error 37 us; skew 500 ppm
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:25.355706 26987 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:25.356477 26987 webserver.cc:460] Webserver started at http://127.3.236.131:41889/ using document root <none> and password file <none>
I20260811 07:09:25.356772 26987 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:25.356861 26987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:25.384964 26987 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260811 07:09:25.386796 27024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.387068 26987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:09:25.387267 26987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f7cd75be07cb4da5b0a1b57171f79850"
format_stamp: "Formatted at 2026-08-11 07:09:24 on dist-test-slave-2d19"
server_key: "a1c8748c8d46a2b42c70c3263d449c0a"
server_key_iv: "f378869eaf9cd926b0df8032eba2e717"
server_key_version: "kuduclusterkey@0"
I20260811 07:09:25.387835 26987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:25.399834 26987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:25.402343 26987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:25.402570 26987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:25.402860 26987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:25.403314 26987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:25.403405 26987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.403478 26987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:25.403525 26987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:25.411475 26987 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:41105
I20260811 07:09:25.411979 26987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:09:25.417706 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 26987
I20260811 07:09:25.420408 27218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:41105 every 8 connection(s)
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:25.428915 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:25.412856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38981 (local address 127.3.236.190:38069)
0811 07:09:25.412906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:25.412908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:25.413183 (+ 275us) server_negotiation.cc:407] Connection header received
0811 07:09:25.413884 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:25.413886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:25.413899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:25.413932 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:25.414778 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.415539 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.420647 (+ 5108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.420955 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.421006 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:25.424426 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:25.424427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:25.424427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:25.424435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:25.427175 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.427683 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.427685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.427685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.427749 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.428059 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.428060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.428061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.428227 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:25.428299 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:25.428725 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 07:09:25.428819 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:09:25.429760 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:25.412866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:38069 (local address 127.3.236.131:38981)
0811 07:09:25.413043 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:25.413091 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:25.413751 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:25.414125 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:25.414129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:25.414362 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:25.414655 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:25.414662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.416128 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.416129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:25.418585 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:25.418589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.421141 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.421142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:25.421298 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:25.422623 (+ 1325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:25.422634 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:25.424308 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:25.427272 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:25.427280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:25.427282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:25.427590 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:25.427823 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:25.427824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:25.427825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:25.427971 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:25.428355 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:25.428362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:25.428542 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 07:09:25.429279 (+ 737us) client_negotiation.cc:240] Negotiation successful
0811 07:09:25.429466 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260811 07:09:25.430536 27225 heartbeater.cc:347] Connected to a master server at 127.3.236.190:38069
I20260811 07:09:25.430651 27225 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.430846 27225 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:25.431456 27225 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:38069 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:09:25.431529 27225 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.431588 27225 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:25.431967 27225 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.432040 27225 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
W20260811 07:09:25.432401 27225 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:09:25.435269 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:25.418332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.3.236.190:38069)
0811 07:09:25.421583 (+ 3251us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:25.421585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:25.421594 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:25.421616 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:25.421617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:25.421629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:25.423317 (+ 1688us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:25.423358 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.424829 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.426341 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:25.430529 (+ 4188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:25.432174 (+ 1645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:25.432232 (+ 58us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:25.432233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:25.432233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:25.432238 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:25.434025 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.434451 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.434452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.434453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.434513 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:25.434745 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:25.434746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:25.434746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:25.434847 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:25.435030 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:25.435068 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 07:09:25.435158 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3219,"thread_start_us":59,"threads_started":1}
Aug 11 07:09:25 dist-test-slave-2d19 krb5kdc[24535](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:25.955808 26749 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:25.955996 26749 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:25.956601 26374 ts_manager.cc:194] Registered new tserver with Master: 1de8d747a6234581b061b13638e9d465 (127.3.236.129:33067)
I20260811 07:09:25.957644 26374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:42239
I20260811 07:09:26.204612 26984 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:26.204813 26984 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:26.205329 26383 ts_manager.cc:194] Registered new tserver with Master: 12a0e4195469440f967b81e84b0db16a (127.3.236.130:40671)
I20260811 07:09:26.205916 26383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:51379
Aug 11 07:09:26 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
I20260811 07:09:26.432569 27225 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:26.432730 27225 heartbeater.cc:511] Master 127.3.236.190:38069 requested a full tablet report, sending...
I20260811 07:09:26.433064 26383 ts_manager.cc:194] Registered new tserver with Master: f7cd75be07cb4da5b0a1b57171f79850 (127.3.236.131:41105)
I20260811 07:09:26.433708 26383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:38981
I20260811 07:09:26.442281 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:09:26.452502 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.443435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.3.236.190:38069)
0811 07:09:26.443589 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.443591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.443697 (+ 106us) server_negotiation.cc:407] Connection header received
0811 07:09:26.443779 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.443780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.443793 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.443871 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:26.444743 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.445616 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.446820 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.447189 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.447261 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.448965 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:26.448967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:26.448967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:26.448975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:26.450888 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:26.451306 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:26.451307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:26.451308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:26.451369 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:26.451558 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:26.451559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:26.451560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:26.451712 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:26.451804 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.452299 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.452389 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260811 07:09:26.455255 26383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:09:26.457602 26383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:09:26.474372 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.460905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:41105 (local address 127.0.0.1:43938)
0811 07:09:26.468373 (+ 7468us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.468395 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:26.468459 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:26.468788 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:26.468792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:26.468795 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:26.468874 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.468876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.470143 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.470144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.471662 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.471664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.474054 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.474055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.474139 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.474154 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:09:26.474225 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:09:26.474280 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7446,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260811 07:09:26.474821 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.461209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.3.236.131:41105)
0811 07:09:26.464365 (+ 3156us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.464370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.468535 (+ 4165us) server_negotiation.cc:407] Connection header received
0811 07:09:26.468582 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.468585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.468603 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.468677 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:26.468964 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.470043 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.471763 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.473860 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.473952 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.474015 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.474698 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.474743 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3116,"thread_start_us":61,"threads_started":1}
I20260811 07:09:26.475792 27069 tablet_service.cc:1467] Processing CreateTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 (DEFAULT_TABLE table=test-table [id=dc79a07cb96f41aea15a2b7b21bcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:26.476183 27069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e13f8824da47d1ae13323798aa2bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:26.478319 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.461455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33067 (local address 127.0.0.1:56208)
0811 07:09:26.469893 (+ 8438us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.469906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:26.469964 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:26.472002 (+ 2038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:26.472003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:26.472006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:26.472088 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.472090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.473891 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.473892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.476537 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.476541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.478067 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.478069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.478141 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.478145 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:26.478200 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:09:26.478242 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8419,"mutex_wait_us":29,"thread_start_us":90,"threads_started":1}
I20260811 07:09:26.478607 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.461727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.3.236.129:33067)
0811 07:09:26.461865 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.461871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.470057 (+ 8186us) server_negotiation.cc:407] Connection header received
0811 07:09:26.470107 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.470110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.470131 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.470213 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:26.472188 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.473786 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.476645 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.477862 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.477953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.478027 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.478482 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.478528 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":51,"threads_started":1}
I20260811 07:09:26.479215 26625 tablet_service.cc:1467] Processing CreateTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 (DEFAULT_TABLE table=test-table [id=dc79a07cb96f41aea15a2b7b21bcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:26.479494 26625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e13f8824da47d1ae13323798aa2bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:26.480162 27283 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Bootstrap starting.
I20260811 07:09:26.480954 27283 tablet_bootstrap.cc:654] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:26.481417 27283 log.cc:824] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:26.481971 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.460905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:40671 (local address 127.0.0.1:45506)
0811 07:09:26.461935 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.461977 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:26.462040 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:26.470588 (+ 8548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:26.470590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:26.470592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:26.470627 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.470629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.471916 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.471917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.473060 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.473062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.481740 (+ 8678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.481741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.481809 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.481814 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:09:26.481844 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 07:09:26.481885 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":156,"threads_started":1}
I20260811 07:09:26.482542 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.461217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45506 (local address 127.3.236.130:40671)
0811 07:09:26.470404 (+ 9187us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.470409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.470421 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:09:26.470461 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.470463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.470482 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.470579 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:26.470674 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.471808 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.480751 (+ 8943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.481631 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.482318 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.482392 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.482420 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.482461 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9144,"thread_start_us":65,"threads_started":1}
I20260811 07:09:26.483392 26856 tablet_service.cc:1467] Processing CreateTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 (DEFAULT_TABLE table=test-table [id=dc79a07cb96f41aea15a2b7b21bcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:26.483795 26856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e13f8824da47d1ae13323798aa2bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:26.483908 27283 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: No bootstrap required, opened a new log
I20260811 07:09:26.483990 27283 ts_tablet_manager.cc:1397] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 07:09:26.488704 27283 raft_consensus.cc:348] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.488886 27283 raft_consensus.cc:374] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:26.488936 27283 raft_consensus.cc:729] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cd75be07cb4da5b0a1b57171f79850, State: Initialized, Role: FOLLOWER
I20260811 07:09:26.489076 27283 consensus_queue.cc:260] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.489418 27283 ts_tablet_manager.cc:1428] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 07:09:26.489549 27225 heartbeater.cc:503] Master 127.3.236.190:38069 was elected leader, sending a full tablet report...
I20260811 07:09:26.489912 27286 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Bootstrap starting.
I20260811 07:09:26.490631 27286 tablet_bootstrap.cc:654] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:26.490991 27286 log.cc:824] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:26.491492 27287 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Bootstrap starting.
I20260811 07:09:26.492532 27287 tablet_bootstrap.cc:654] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:26.493044 27287 log.cc:824] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:26.493896 27286 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: No bootstrap required, opened a new log
I20260811 07:09:26.494092 27286 ts_tablet_manager.cc:1397] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:09:26.494416 27287 tablet_bootstrap.cc:492] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: No bootstrap required, opened a new log
I20260811 07:09:26.494694 27287 ts_tablet_manager.cc:1397] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:09:26.497510 27286 raft_consensus.cc:348] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.497756 27286 raft_consensus.cc:374] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:26.497836 27286 raft_consensus.cc:729] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12a0e4195469440f967b81e84b0db16a, State: Initialized, Role: FOLLOWER
I20260811 07:09:26.498044 27286 consensus_queue.cc:260] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.498368 27286 ts_tablet_manager.cc:1428] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 07:09:26.498529 26984 heartbeater.cc:503] Master 127.3.236.190:38069 was elected leader, sending a full tablet report...
I20260811 07:09:26.500574 27287 raft_consensus.cc:348] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.500886 27287 raft_consensus.cc:374] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:26.501008 27287 raft_consensus.cc:729] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de8d747a6234581b061b13638e9d465, State: Initialized, Role: FOLLOWER
I20260811 07:09:26.501220 27287 consensus_queue.cc:260] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.501705 26749 heartbeater.cc:503] Master 127.3.236.190:38069 was elected leader, sending a full tablet report...
I20260811 07:09:26.501713 27287 ts_tablet_manager.cc:1428] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260811 07:09:26.663553 27231 tablet.cc:2369] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 07:09:26 dist-test-slave-2d19 krb5kdc[24535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432165, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for HTTP/127.3.236.148@KRBTEST.COM
W20260811 07:09:26.689204 26750 tablet.cc:2369] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:09:26.690784 26985 tablet.cc:2369] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:09:26.719321 27290 raft_consensus.cc:482] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:26.719499 27290 raft_consensus.cc:504] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.719933 27290 leader_election.cc:288] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12a0e4195469440f967b81e84b0db16a (127.3.236.130:40671), 1de8d747a6234581b061b13638e9d465 (127.3.236.129:33067)
I20260811 07:09:26.726569 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.720298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:52085 (local address 127.3.236.130:40671)
0811 07:09:26.720365 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.720367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.720728 (+ 361us) server_negotiation.cc:407] Connection header received
0811 07:09:26.720804 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.720806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.720818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.720866 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:26.721170 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.721895 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.725554 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.726122 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.726178 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.726187 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.726443 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.726495 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:09:26.726698 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.720300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:55287 (local address 127.3.236.129:33067)
0811 07:09:26.720405 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.720407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.721357 (+ 950us) server_negotiation.cc:407] Connection header received
0811 07:09:26.721457 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.721459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.721468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.721512 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:26.721714 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.722428 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.725155 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.725735 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.725784 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.725793 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.726582 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.726627 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 07:09:26.727170 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.720109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:40671 (local address 127.3.236.131:52085)
0811 07:09:26.720630 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.720642 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:26.720740 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:26.720966 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:26.720968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:26.720969 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:26.721074 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.721077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.722592 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.722593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.725447 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.725451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.726257 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.726258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.726348 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.726352 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:26.727023 (+ 671us) client_negotiation.cc:240] Negotiation successful
0811 07:09:26.727072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":456,"threads_started":1}
I20260811 07:09:26.727710 26885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e13f8824da47d1ae13323798aa2bf7" candidate_uuid: "f7cd75be07cb4da5b0a1b57171f79850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a0e4195469440f967b81e84b0db16a" is_pre_election: true
I20260811 07:09:26.727916 26885 raft_consensus.cc:2454] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd75be07cb4da5b0a1b57171f79850 in term 0.
I20260811 07:09:26.728312 27030 leader_election.cc:302] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12a0e4195469440f967b81e84b0db16a, f7cd75be07cb4da5b0a1b57171f79850; no voters:
I20260811 07:09:26.728968 27290 raft_consensus.cc:2781] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:09:26.729048 27290 raft_consensus.cc:482] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:09:26.729071 27290 raft_consensus.cc:3037] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:26.729969 27290 raft_consensus.cc:504] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.730118 27290 leader_election.cc:288] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [CANDIDATE]: Term 1 election: Requested vote from peers 12a0e4195469440f967b81e84b0db16a (127.3.236.130:40671), 1de8d747a6234581b061b13638e9d465 (127.3.236.129:33067)
I20260811 07:09:26.731240 26885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e13f8824da47d1ae13323798aa2bf7" candidate_uuid: "f7cd75be07cb4da5b0a1b57171f79850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a0e4195469440f967b81e84b0db16a"
I20260811 07:09:26.731324 26885 raft_consensus.cc:3037] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:26.732234 26885 raft_consensus.cc:2454] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd75be07cb4da5b0a1b57171f79850 in term 1.
I20260811 07:09:26.732478 27030 leader_election.cc:302] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12a0e4195469440f967b81e84b0db16a, f7cd75be07cb4da5b0a1b57171f79850; no voters:
I20260811 07:09:26.732959 27290 raft_consensus.cc:2781] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:26.733122 27290 raft_consensus.cc:686] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 1 LEADER]: Becoming Leader. State: Replica: f7cd75be07cb4da5b0a1b57171f79850, State: Running, Role: LEADER
I20260811 07:09:26.733214 27290 consensus_queue.cc:237] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } }
I20260811 07:09:26.734177 26383 catalog_manager.cc:5495] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7cd75be07cb4da5b0a1b57171f79850 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "f7cd75be07cb4da5b0a1b57171f79850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7cd75be07cb4da5b0a1b57171f79850" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 41105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 } health_report { overall_health: UNKNOWN } } }
I20260811 07:09:26.734428 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.720136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33067 (local address 127.3.236.131:55287)
0811 07:09:26.721291 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.721302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:26.721385 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:26.721577 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:26.721578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:26.721579 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:26.721646 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.721647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.722902 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.722902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.725052 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:26.725056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.726204 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.726205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:26.726275 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.726279 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:26.726822 (+ 543us) client_negotiation.cc:240] Negotiation successful
0811 07:09:26.726865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"thread_start_us":439,"threads_started":1}
I20260811 07:09:26.735321 26656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e13f8824da47d1ae13323798aa2bf7" candidate_uuid: "f7cd75be07cb4da5b0a1b57171f79850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de8d747a6234581b061b13638e9d465" is_pre_election: true
I20260811 07:09:26.735502 26656 raft_consensus.cc:2454] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cd75be07cb4da5b0a1b57171f79850 in term 0.
I20260811 07:09:26.735651 26655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e13f8824da47d1ae13323798aa2bf7" candidate_uuid: "f7cd75be07cb4da5b0a1b57171f79850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de8d747a6234581b061b13638e9d465"
I20260811 07:09:26.735725 26655 raft_consensus.cc:3037] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:26.736531 26655 raft_consensus.cc:2454] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cd75be07cb4da5b0a1b57171f79850 in term 1.
I20260811 07:09:26.753527 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.750001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.3.236.131:41105)
0811 07:09:26.750057 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:26.750059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:26.750068 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:26.750114 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:26.750115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:26.750141 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:26.750219 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:09:26.750431 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.751155 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.752361 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:26.752636 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:26.752707 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:26.752941 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:09:26.753082 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:09:26.753169 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:26.753422 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 07:09:26.753445 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:09:26.762064 26655 raft_consensus.cc:1261] T 78e13f8824da47d1ae13323798aa2bf7 P 1de8d747a6234581b061b13638e9d465 [term 1 FOLLOWER]: Refusing update from remote peer f7cd75be07cb4da5b0a1b57171f79850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:26.762650 27307 consensus_queue.cc:1035] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de8d747a6234581b061b13638e9d465" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:26.768221 26885 raft_consensus.cc:1261] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 1 FOLLOWER]: Refusing update from remote peer f7cd75be07cb4da5b0a1b57171f79850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:26.768591 27307 consensus_queue.cc:1035] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12a0e4195469440f967b81e84b0db16a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 40671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:26.773325 27311 mvcc.cc:205] Tried to move back new op lower bound from 7317226155043815424 to 7317226154939650048. Current Snapshot: MvccSnapshot[applied={T|T < 7317226155043815424}]
I20260811 07:09:26.773512 27314 mvcc.cc:205] Tried to move back new op lower bound from 7317226155043815424 to 7317226154939650048. Current Snapshot: MvccSnapshot[applied={T|T < 7317226155043815424}]
I20260811 07:09:26.777173 26383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:26.777275 26383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:26.778209 26383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d3630a5111f40fe8247a365e071fcb5: Sending DeleteTablet for 3 replicas of tablet 78e13f8824da47d1ae13323798aa2bf7
I20260811 07:09:26.778714 26856 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:26 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45506
I20260811 07:09:26.779007 27069 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:26 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:43938
I20260811 07:09:26.779443 26625 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e13f8824da47d1ae13323798aa2bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:26 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:56208
I20260811 07:09:26.780164 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 26449
I20260811 07:09:26.789251 27318 mvcc.cc:205] Tried to move back new op lower bound from 7317226155043815424 to 7317226154939650048. Current Snapshot: MvccSnapshot[applied={T|T < 7317226155043815424}]
I20260811 07:09:26.796604 27329 tablet_replica.cc:330] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: stopping tablet replica
I20260811 07:09:26.796859 27330 tablet_replica.cc:330] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: stopping tablet replica
I20260811 07:09:26.797030 27330 raft_consensus.cc:2229] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:09:26.797191 27329 raft_consensus.cc:2229] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:26.797422 27330 raft_consensus.cc:2258] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:26.797612 27329 raft_consensus.cc:2258] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:26.803104 27330 ts_tablet_manager.cc:1905] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:26.803489 27329 ts_tablet_manager.cc:1905] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:26.806917 27330 ts_tablet_manager.cc:1918] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:26.807125 27329 ts_tablet_manager.cc:1918] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:26.807266 27330 log.cc:1196] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/78e13f8824da47d1ae13323798aa2bf7
I20260811 07:09:26.807406 27329 log.cc:1196] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/78e13f8824da47d1ae13323798aa2bf7
I20260811 07:09:26.807817 27330 ts_tablet_manager.cc:1939] T 78e13f8824da47d1ae13323798aa2bf7 P f7cd75be07cb4da5b0a1b57171f79850: Deleting consensus metadata
I20260811 07:09:26.808280 27329 ts_tablet_manager.cc:1939] T 78e13f8824da47d1ae13323798aa2bf7 P 12a0e4195469440f967b81e84b0db16a: Deleting consensus metadata
I20260811 07:09:26.809046 26341 catalog_manager.cc:4841] TS 12a0e4195469440f967b81e84b0db16a (127.3.236.130:40671): tablet 78e13f8824da47d1ae13323798aa2bf7 (table test-table [id=dc79a07cb96f41aea15a2b7b21bcf041]) successfully deleted
I20260811 07:09:26.809535 26340 catalog_manager.cc:4841] TS f7cd75be07cb4da5b0a1b57171f79850 (127.3.236.131:41105): tablet 78e13f8824da47d1ae13323798aa2bf7 (table test-table [id=dc79a07cb96f41aea15a2b7b21bcf041]) successfully deleted
W20260811 07:09:26.820636 26338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:33067 (error 108)
I20260811 07:09:26.822343 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.822172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33067 (local address 127.0.0.1:56212)
0811 07:09:26.822236 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.822298 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 07:09:26.822337 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 26758
W20260811 07:09:26.825450 26338 catalog_manager.cc:4584] TS 1de8d747a6234581b061b13638e9d465 (127.3.236.129:33067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78e13f8824da47d1ae13323798aa2bf7: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33067: connect: Connection refused (error 111)
I20260811 07:09:26.878130 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.877979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33067 (local address 127.0.0.1:56224)
0811 07:09:26.878043 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.878080 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 07:09:26.903302 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 26987
I20260811 07:09:26.911546 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:26.911388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:33067 (local address 127.0.0.1:56226)
0811 07:09:26.911451 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:26.911485 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 07:09:26.930819 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 26307
I20260811 07:09:26.945629 4018 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 07:09:27.300413 4018 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 07:09:27.300508 4018 mini_ranger.cc:66] Stopping Ranger...
I20260811 07:09:27.655203 4018 mini_ranger.cc:68] Stopped Ranger
2026-08-11 07:09:27.655 UTC [24593] LOG: received smart shutdown request
2026-08-11 07:09:27.660 UTC [24593] LOG: background worker "logical replication launcher" (PID 24608) exited with exit code 1
2026-08-11 07:09:27.660 UTC [24599] LOG: shutting down
2026-08-11 07:09:27.828 UTC [24593] LOG: database system is shut down
2026-08-11T07:09:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49983 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:09:28 dist-test-slave-2d19 krb5kdc[27346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:09:28 dist-test-slave-2d19 krb5kdc[27346](info): set up 2 sockets
Aug 11 07:09:28 dist-test-slave-2d19 krb5kdc[27346](info): commencing operation
krb5kdc: starting...
W20260811 07:09:29.308532 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:09:29.436061 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:32839
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:29.520573 27377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:29.520784 27377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:29.520807 27377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:29.521941 27377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:29.521981 27377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:29.521996 27377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:29.522032 27377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:29.522063 27377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:29.523568 27377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:29.523821 27377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:29.524055 27377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:29.528913 27389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.528930 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.528904 27387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:29.529228 27377 server_base.cc:1034] running on GCE node
I20260811 07:09:29.529517 27377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:29.529798 27377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:29.530969 27377 hybrid_clock.cc:648] HybridClock initialized: now 1786432169530963 us; error 36 us; skew 500 ppm
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:29.533488 27377 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:29.533957 27377 webserver.cc:460] Webserver started at http://127.3.236.190:38479/ using document root <none> and password file <none>
I20260811 07:09:29.534192 27377 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:29.534260 27377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:29.534412 27377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:29.535440 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "3c62b7c927a54444a32d19bb1028f69d"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "27f5cf3b7051e020ec3196331e12b9a6"
server_key_iv: "24d0eeeea596bad1eb5dc0bd04a21ef5"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.535825 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "3c62b7c927a54444a32d19bb1028f69d"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "27f5cf3b7051e020ec3196331e12b9a6"
server_key_iv: "24d0eeeea596bad1eb5dc0bd04a21ef5"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.537214 27377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:09:29.537992 27400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:29.538450 27377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:29.538604 27377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "3c62b7c927a54444a32d19bb1028f69d"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "27f5cf3b7051e020ec3196331e12b9a6"
server_key_iv: "24d0eeeea596bad1eb5dc0bd04a21ef5"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.538686 27377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:29.549553 27377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:29.551299 27377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:29.551411 27377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:29.555512 27377 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:32839
I20260811 07:09:29.555572 27502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:32839 every 8 connection(s)
I20260811 07:09:29.555917 27377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:29.556855 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:29.558457 27503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:09:29.561376 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 27377
I20260811 07:09:29.561519 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:09:29.561765 4018 external_mini_cluster.cc:1428] Setting key 0ddfe5115a7bca0ac61bbc193438938c
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:29.568728 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.559158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:32839 (local address 127.0.0.1:43852)
0811 07:09:29.559321 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.559356 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.559789 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.560070 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.560072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.560393 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.560552 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.560557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.561595 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.561596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.562695 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.562699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.563186 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.563188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.563276 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.563876 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.563885 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.565309 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.567133 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.567140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.567142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.567343 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.567605 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.567606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.567606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.567697 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.568090 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.568095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.568159 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.568290 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.568535 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":99,"thread_start_us":64,"threads_started":1}
I20260811 07:09:29.568733 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.559306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.3.236.190:32839)
0811 07:09:29.559436 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.559440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.559468 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:09:29.559921 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.559923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.559940 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.560017 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.560640 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.561485 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.562779 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.563095 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.563175 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.565400 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.565408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.565409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.565416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.567027 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.567430 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.567434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.567435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.567525 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.567801 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.567803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.567803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.568009 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.568098 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.568332 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.568469 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260811 07:09:29.569475 27503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:09:29.569868 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.569743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42802)
0811 07:09:29.569794 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.569839 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:09:29.570009 27402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.236.189@KRBTEST.COM; defaulting to no policy
I20260811 07:09:29.570384 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.570310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42806)
0811 07:09:29.570338 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.570362 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260811 07:09:29.570490 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36665: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
I20260811 07:09:29.589183 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:36665
--webserver_interface=127.3.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:29.619796 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } attempt: 1
I20260811 07:09:29.620412 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.620239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42820)
0811 07:09:29.620335 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.620382 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 07:09:29.620786 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.620705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42832)
0811 07:09:29.620746 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.620771 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 07:09:29.620891 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36665: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
W20260811 07:09:29.671627 27518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:29.671862 27518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:29.671917 27518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:29.673246 27518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:29.673312 27518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:29.673336 27518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:29.673358 27518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:29.673385 27518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:29.674885 27518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:29.675171 27518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:29.675400 27518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:29.680274 27529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.680297 27533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.680280 27530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:29.680477 27518 server_base.cc:1034] running on GCE node
I20260811 07:09:29.680759 27518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:29.681010 27518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:29.682154 27518 hybrid_clock.cc:648] HybridClock initialized: now 1786432169682144 us; error 32 us; skew 500 ppm
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:29.684752 27518 init.cc:377] Logged in from keytab as kudu/127.3.236.189@KRBTEST.COM (short username kudu)
I20260811 07:09:29.685137 27518 webserver.cc:460] Webserver started at http://127.3.236.189:37027/ using document root <none> and password file <none>
I20260811 07:09:29.685309 27518 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:29.685348 27518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:29.685448 27518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:29.686439 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data/instance:
uuid: "073a6e30a339471ebbe9b03920fc415f"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "898b63a98c0bdcf07489460fa8574a7d"
server_key_iv: "1e639e05ec221aa049fe430c9ee1ae81"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.686751 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal/instance:
uuid: "073a6e30a339471ebbe9b03920fc415f"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "898b63a98c0bdcf07489460fa8574a7d"
server_key_iv: "1e639e05ec221aa049fe430c9ee1ae81"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.688050 27518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:09:29.688850 27546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:29.689040 27518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:29.689030 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } attempt: 2
I20260811 07:09:29.689136 27518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "073a6e30a339471ebbe9b03920fc415f"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "898b63a98c0bdcf07489460fa8574a7d"
server_key_iv: "1e639e05ec221aa049fe430c9ee1ae81"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.689216 27518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:29.689414 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.689287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42836)
0811 07:09:29.689335 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.689375 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 07:09:29.689764 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.689668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42850)
0811 07:09:29.689714 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.689741 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:09:29.689880 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36665: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36665: connect: Connection refused (error 111)
I20260811 07:09:29.707840 27518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:29.709930 27518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:29.710075 27518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:29.714210 27518 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.189:36665
I20260811 07:09:29.714262 27635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.189:36665 every 8 connection(s)
I20260811 07:09:29.714629 27518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
I20260811 07:09:29.715502 27636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:29.717079 27636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:29.724457 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 27518
I20260811 07:09:29.724597 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-1/wal/instance
I20260811 07:09:29.724799 4018 external_mini_cluster.cc:1428] Setting key a3a14983a621f6da5ea36c25827d6057
I20260811 07:09:29.725862 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.717797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.3.236.190:32839)
0811 07:09:29.717876 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.717879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.718077 (+ 198us) server_negotiation.cc:407] Connection header received
0811 07:09:29.718576 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.718577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.718592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.718657 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.719263 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.719746 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.720816 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.721027 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.721114 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.723101 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.723103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.723104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.723112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.724377 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.724858 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.724859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.724860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.724923 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.725189 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.725190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.725190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.725322 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.725391 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.725683 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.725775 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:09:29.726032 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.717756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:32839 (local address 127.0.0.1:43866)
0811 07:09:29.717943 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.717981 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.718457 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.718732 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.718735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.718972 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.719161 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.719165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.719846 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.719848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.720721 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.720724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.721128 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.721129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.721253 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.721676 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.721684 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.723004 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.724456 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.724463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.724466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.724782 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.724991 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.724991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.724992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.725112 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.725396 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.725404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.725543 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.725647 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.725801 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260811 07:09:29.726742 27636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.188@KRBTEST.COM; defaulting to no policy
I20260811 07:09:29.734261 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.727047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42866)
0811 07:09:29.727266 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.727278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.727334 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.727496 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.727497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.727654 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.727712 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.727714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.728622 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.728622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.729133 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.729134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.729791 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.729792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.729845 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.730237 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.730241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.731409 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.733174 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.733175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.733176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.733345 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.733573 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.733573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.733574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.733635 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.733912 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.733913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.733952 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.734025 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.734127 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 07:09:29.734328 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.727120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.3.236.189:36665)
0811 07:09:29.727278 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.727282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.727326 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:09:29.727391 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.727393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.727403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.727465 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.727787 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.728543 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.729218 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.729706 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.729795 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.731503 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.731511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.731511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.731518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.733071 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.733441 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.733445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.733445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.733506 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.733699 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.733700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.733700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.733842 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.733935 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.734054 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.734201 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260811 07:09:29.735077 27636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:09:29.735466 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.735340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34652)
0811 07:09:29.735389 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.735438 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 07:09:29.735685 27547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
I20260811 07:09:29.736191 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.736106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34654)
0811 07:09:29.736147 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.736172 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 07:09:29.736356 27636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
I20260811 07:09:29.754626 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:38791
--webserver_interface=127.3.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:29.781040 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } attempt: 3
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:09:29.785627 27636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 1
I20260811 07:09:29.786089 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.785886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34668)
0811 07:09:29.786029 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.786058 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 07:09:29.786680 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.786383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34678)
0811 07:09:29.786628 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.786654 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227}
W20260811 07:09:29.786798 27636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
I20260811 07:09:29.788408 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.781447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42872)
0811 07:09:29.781517 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.781527 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.781589 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.781748 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.781749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.781919 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.781986 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.781988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.782678 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.782679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.783217 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.783218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.783602 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.783603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.783690 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.784150 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.784163 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.785422 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.787441 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.787442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.787442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.787609 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.787817 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.787818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.787818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.787869 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.788131 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.788132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.788169 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.788219 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.788345 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:09:29.788417 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.781531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.3.236.189:36665)
0811 07:09:29.781599 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.781602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.781616 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:09:29.781663 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.781664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.781673 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.781729 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.782079 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.782590 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.783307 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.783524 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.783602 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.785529 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.785530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.785530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.785543 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.787362 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.787686 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.787687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.787687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.787751 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.787928 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.787928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.787929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.788068 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.788136 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.788242 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.788335 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:09:29.788841 27503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:09:29.789187 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.789084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34694)
0811 07:09:29.789136 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.789162 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 07:09:29.789565 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.789476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34696)
0811 07:09:29.789520 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.789544 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 07:09:29.789657 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
I20260811 07:09:29.820796 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 1
I20260811 07:09:29.821360 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.821213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34704)
0811 07:09:29.821285 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.821328 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:09:29.821735 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.821633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34720)
0811 07:09:29.821681 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.821710 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 07:09:29.821831 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
W20260811 07:09:29.840204 27650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:29.840485 27650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:29.840524 27650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:29.841676 27650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:29.841746 27650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:29.841770 27650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:29.841791 27650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:29.841821 27650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:29.843384 27650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:32839,127.3.236.189:36665,127.3.236.188:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:38791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:29.843670 27650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:29.843897 27650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:29.849174 27657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.849152 27659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:29.849156 27669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:29.849738 27650 server_base.cc:1034] running on GCE node
I20260811 07:09:29.850020 27650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:29.850289 27650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:29.851464 27650 hybrid_clock.cc:648] HybridClock initialized: now 1786432169851443 us; error 31 us; skew 500 ppm
I20260811 07:09:29.854960 27636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 2
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:29.855634 27650 init.cc:377] Logged in from keytab as kudu/127.3.236.188@KRBTEST.COM (short username kudu)
I20260811 07:09:29.855674 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.855537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34734)
0811 07:09:29.855601 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.855630 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:09:29.856010 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.855923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34736)
0811 07:09:29.855962 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.855985 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 07:09:29.856277 27650 webserver.cc:460] Webserver started at http://127.3.236.188:43405/ using document root <none> and password file <none>
I20260811 07:09:29.856614 27650 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:29.856721 27650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:29.856886 27650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:29.858596 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data/instance:
uuid: "abf6a03526eb429fb4f48f000eda654c"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "0e16c8bb3329a236c99a25812fd97d94"
server_key_iv: "7b820a308be69d6cf975ee955bbd99c7"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.858983 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal/instance:
uuid: "abf6a03526eb429fb4f48f000eda654c"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "0e16c8bb3329a236c99a25812fd97d94"
server_key_iv: "7b820a308be69d6cf975ee955bbd99c7"
server_key_version: "encryptionkey@0"
W20260811 07:09:29.860508 27636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
I20260811 07:09:29.860970 27650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 07:09:29.863045 27679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:29.863260 27650 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:09:29.863379 27650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "abf6a03526eb429fb4f48f000eda654c"
format_stamp: "Formatted at 2026-08-11 07:09:29 on dist-test-slave-2d19"
server_key: "0e16c8bb3329a236c99a25812fd97d94"
server_key_iv: "7b820a308be69d6cf975ee955bbd99c7"
server_key_version: "encryptionkey@0"
I20260811 07:09:29.863473 27650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:29.893532 27650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:29.895648 27650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:29.895840 27650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:29.896963 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 2
I20260811 07:09:29.897352 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.897230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34748)
0811 07:09:29.897288 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.897324 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:09:29.897722 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.897622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34758)
0811 07:09:29.897671 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.897701 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 07:09:29.897827 27503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38791: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38791: connect: Connection refused (error 111)
I20260811 07:09:29.900818 27650 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.188:38791
I20260811 07:09:29.900880 27770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.188:38791 every 8 connection(s)
I20260811 07:09:29.901279 27650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
I20260811 07:09:29.902215 27771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:29.903824 27771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:29.911357 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 27650
I20260811 07:09:29.911505 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786432006544353-4018-0/minicluster-data/master-2/wal/instance
I20260811 07:09:29.911765 4018 external_mini_cluster.cc:1428] Setting key 243ce2911903881ce3b00fab05f357be
I20260811 07:09:29.913117 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.904561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:32839 (local address 127.0.0.1:43880)
0811 07:09:29.904720 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.904762 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.905274 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.905515 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.905518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.905759 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.905958 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.905962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.906563 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.906564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.907383 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.907385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.907724 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.907725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.907823 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.908299 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.908316 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.909635 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.911057 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.911064 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.911065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.911377 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.911668 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.911669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.911670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.911813 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.912580 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.912584 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.912743 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.912839 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.912958 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260811 07:09:29.913112 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.904600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.3.236.190:32839)
0811 07:09:29.904656 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.904658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.904847 (+ 189us) server_negotiation.cc:407] Connection header received
0811 07:09:29.905397 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.905398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.905408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.905460 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.906029 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.906487 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.907459 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.907652 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.907704 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.909719 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.909720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.909721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.909727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.910963 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.911508 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.911509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.911509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.911590 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.912332 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.912334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.912334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.912504 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.912576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.912904 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.912993 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:09:29.913793 27771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:09:29.921461 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.914089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.3.236.189:36665)
0811 07:09:29.914131 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.914133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.914147 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:09:29.914169 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.914170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.914177 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.914224 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.914505 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.915214 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.916143 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.916512 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.916596 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.918675 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.918676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.918677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.918700 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.920126 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920587 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.920588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.920589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.920654 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920873 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.920874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.920874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.921011 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.921086 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.921261 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.921350 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:09:29.921411 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.914022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36665 (local address 127.0.0.1:42878)
0811 07:09:29.914082 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.914091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.914121 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.914246 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.914247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.914379 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.914429 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.914431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.915303 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.915304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.916045 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.916048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.916593 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.916594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.916645 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.917037 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.917041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.918588 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.920242 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.920243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.920504 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.920733 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.920734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.920802 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.921081 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.921081 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.921116 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.921203 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.921297 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:09:29.921712 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.912248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.3.236.188:38791)
0811 07:09:29.912431 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.912435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.912469 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:09:29.912536 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.912538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.912570 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.912655 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.913407 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.914379 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.915445 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.915876 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.915954 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.918329 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.918339 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.918339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.918348 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.920158 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920595 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.920598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.920598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.920654 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.920885 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.920885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.920886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.921094 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.921271 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.921332 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.921415 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260811 07:09:29.922142 27771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:29.929019 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.922392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34786)
0811 07:09:29.922431 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.922441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.922477 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.922606 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.922607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.922783 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.922822 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.922825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.923655 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.923655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.924353 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.924355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.924715 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.924720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.924757 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.925129 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.925133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.926215 (+ 1082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.927954 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.927954 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.927955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.928172 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.928427 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.928428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.928428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.928503 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.928730 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.928731 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.928775 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.928864 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.928958 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 07:09:29.929142 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.922461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.3.236.188:38791)
0811 07:09:29.922495 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.922496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.922503 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:09:29.922527 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.922527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.922538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.922602 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.922900 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.923576 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.924428 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.924620 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.924703 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.926313 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.926314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.926314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.926321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.927873 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.928296 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.928297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.928297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.928354 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.928573 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.928574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.928574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.928665 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.928714 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.928907 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.929018 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":17}
I20260811 07:09:29.930099 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.922498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.3.236.190:32839)
0811 07:09:29.922535 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.922537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.922569 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:09:29.922590 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.922591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.922601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.922648 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.923172 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.923809 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.924683 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.924904 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.924956 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.927136 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.927138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.927138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.927146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.928752 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.929286 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.929287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.929287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.929342 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.929518 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.929518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.929519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.929652 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.929868 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.929910 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.929988 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:09:29.931016 27771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: Bootstrap starting.
I20260811 07:09:29.932050 27771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:29.932406 27771 log.cc:824] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:29.933244 27771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: No bootstrap required, opened a new log
I20260811 07:09:29.934816 27771 raft_consensus.cc:348] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:29.934993 27771 raft_consensus.cc:374] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:29.935039 27771 raft_consensus.cc:729] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf6a03526eb429fb4f48f000eda654c, State: Initialized, Role: FOLLOWER
I20260811 07:09:29.935168 27771 consensus_queue.cc:260] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:09:29.935446 27782 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:29.935508 27771 sys_catalog.cc:564] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:29.935570 27782 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:09:29.937415 27805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:09:29.937470 27805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:09:29.938668 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.931068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.3.236.189:36665)
0811 07:09:29.931110 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.931113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.931138 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:09:29.931160 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.931161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.931171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.931221 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.931652 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.932368 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.933081 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.933354 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.933430 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.935738 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.935739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.935740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.935746 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.937504 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.937841 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.937842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.937843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.937903 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.938129 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.938130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.938130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.938259 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.938451 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.938500 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.938574 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:09:29.940213 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:09:29.938061 27650 master_runner.cc:186] Error getting master registration for 127.3.236.190:32839: OK, instance_id { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" instance_seqno: 1786432169554363 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:09:29.940604 27650 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:09:29.949864 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.941822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34800 (local address 127.3.236.188:38791)
0811 07:09:29.941855 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.941856 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.941865 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:29.941886 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.941887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.941911 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.941972 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.942774 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.943548 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.944657 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.944948 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.944988 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.946761 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.946763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.946764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.946771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.948506 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.948905 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.948906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.948907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.948962 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.949183 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.949184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.949184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.949365 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.949457 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.949680 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.949763 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:09:29.950444 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.941602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43890 (local address 127.3.236.190:32839)
0811 07:09:29.941791 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.941793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.942158 (+ 365us) server_negotiation.cc:407] Connection header received
0811 07:09:29.942253 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.942254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.942264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.942293 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.943067 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.943774 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.944804 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.945091 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.945149 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.946817 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.946818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.946818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.946825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.948911 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.949289 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.949290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.949290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.949343 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.949574 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.949575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.949576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.949706 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.949774 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.950274 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.950349 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 07:09:29.951331 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.941870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.3.236.189:36665)
0811 07:09:29.941911 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.941913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.942164 (+ 251us) server_negotiation.cc:407] Connection header received
0811 07:09:29.942263 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.942263 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.942274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.942304 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.942936 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.943610 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.944656 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.944948 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.944988 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.947238 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.947239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.947240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.947247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.949104 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.949579 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.949580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.949581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.949639 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.949841 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.949842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.949842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.949964 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.950575 (+ 611us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.951205 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.951271 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:09:29.951676 27636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 3
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:09:29.958699 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.951919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34814)
0811 07:09:29.951966 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.951976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.952011 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.952121 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.952122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.952296 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.952335 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.952337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.952890 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.952891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.953684 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.953686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.954057 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.954058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.954115 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.954470 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:29.954490 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:29.955988 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:29.957604 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.957605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.957606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.957796 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.958031 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:29.958032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:29.958032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:29.958091 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:29.958344 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:29.958345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:29.958410 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 07:09:29.958506 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 07:09:29.958606 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:09:29.958696 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.951985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.3.236.188:38791)
0811 07:09:29.952025 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.952027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.952034 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:09:29.952055 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.952056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.952064 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.952117 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.952393 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.952816 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.953768 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.953981 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.954050 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:29.956077 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:29.956078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:29.956078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:29.956087 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:29.957498 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.957884 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.957885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.957886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.957941 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:29.958164 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:29.958165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:29.958165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:29.958275 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:29.958350 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:29.958533 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 07:09:29.958606 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:09:29.960587 27636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: Bootstrap starting.
I20260811 07:09:29.961265 27636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:29.961570 27636 log.cc:824] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:29.962363 27636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: No bootstrap required, opened a new log
I20260811 07:09:29.963784 27636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:29.963897 27636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:29.963920 27636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073a6e30a339471ebbe9b03920fc415f, State: Initialized, Role: FOLLOWER
I20260811 07:09:29.964077 27636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:29.964356 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:29.964432 27636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:29.964437 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:09:29.966183 27846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:09:29.966296 27846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:09:29.966337 27518 master_runner.cc:186] Error getting master registration for 127.3.236.190:32839: OK, instance_id { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" instance_seqno: 1786432169554363 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:09:29.969461 27518 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:09:29.996978 27503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } attempt: 3
Aug 11 07:09:29 dist-test-slave-2d19 krb5kdc[27346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:09:30.005894 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.997397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.3.236.188:38791)
0811 07:09:29.997467 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:29.997470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:29.997479 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:29.997506 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:29.997507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:29.997518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:29.997590 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:29.997963 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.998735 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.999848 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:30.000171 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:30.000266 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:30.002750 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:30.002752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:30.002752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:30.002760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:30.004572 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:30.005013 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:30.005015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:30.005015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:30.005082 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:30.005308 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:30.005309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:30.005310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:30.005452 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:30.005524 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:30.005710 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 07:09:30.005792 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:09:30.005894 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:29.997325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38791 (local address 127.0.0.1:34822)
0811 07:09:29.997382 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:29.997394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:29.997433 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:29.997597 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:29.997598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:29.997800 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:29.997856 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.997858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:29.998833 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:29.998834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:29.999740 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:29.999744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:30.000290 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:30.000292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:30.000355 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:30.000932 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:30.000938 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:30.002645 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:30.004664 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:30.004666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:30.004666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:30.004929 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:30.005162 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:30.005163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:30.005163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:30.005239 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:30.005533 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:30.005534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:30.005588 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 07:09:30.005672 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:09:30.005777 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 07:09:30.008237 27503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: Bootstrap starting.
I20260811 07:09:30.009142 27503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:30.009498 27503 log.cc:824] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:30.010581 27503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: No bootstrap required, opened a new log
I20260811 07:09:30.013067 27503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:30.014513 27503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:30.014645 27503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c62b7c927a54444a32d19bb1028f69d, State: Initialized, Role: FOLLOWER
I20260811 07:09:30.014887 27503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:30.015336 27851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.015452 27851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:09:30.015372 27503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 07:09:30.017663 27874 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:09:30.017724 27874 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:09:30.020303 27377 master_runner.cc:263] No leader master found from master 3c62b7c927a54444a32d19bb1028f69d
I20260811 07:09:30.020356 27377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 07:09:30.052170 27408 tablet.cc:2369] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:09:30.067411 27851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:30.067559 27851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:30.067847 27851 leader_election.cc:288] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073a6e30a339471ebbe9b03920fc415f (127.3.236.189:36665), abf6a03526eb429fb4f48f000eda654c (127.3.236.188:38791)
I20260811 07:09:30.068426 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c62b7c927a54444a32d19bb1028f69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf6a03526eb429fb4f48f000eda654c" is_pre_election: true
I20260811 07:09:30.068602 27719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c62b7c927a54444a32d19bb1028f69d in term 0.
I20260811 07:09:30.069108 27401 leader_election.cc:302] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c62b7c927a54444a32d19bb1028f69d, abf6a03526eb429fb4f48f000eda654c; no voters:
I20260811 07:09:30.069315 27851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:09:30.069391 27851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:09:30.069419 27851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:30.070464 27851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:30.070595 27851 leader_election.cc:288] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [CANDIDATE]: Term 1 election: Requested vote from peers 073a6e30a339471ebbe9b03920fc415f (127.3.236.189:36665), abf6a03526eb429fb4f48f000eda654c (127.3.236.188:38791)
I20260811 07:09:30.071188 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c62b7c927a54444a32d19bb1028f69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf6a03526eb429fb4f48f000eda654c"
I20260811 07:09:30.071277 27719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:30.072978 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c62b7c927a54444a32d19bb1028f69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073a6e30a339471ebbe9b03920fc415f"
I20260811 07:09:30.073140 27587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:30.073815 27719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c62b7c927a54444a32d19bb1028f69d in term 1.
I20260811 07:09:30.074070 27401 leader_election.cc:302] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c62b7c927a54444a32d19bb1028f69d, abf6a03526eb429fb4f48f000eda654c; no voters:
I20260811 07:09:30.074280 27851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:30.074419 27587 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c62b7c927a54444a32d19bb1028f69d in term 1.
I20260811 07:09:30.074436 27851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [term 1 LEADER]: Becoming Leader. State: Replica: 3c62b7c927a54444a32d19bb1028f69d, State: Running, Role: LEADER
I20260811 07:09:30.072991 27589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c62b7c927a54444a32d19bb1028f69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073a6e30a339471ebbe9b03920fc415f" is_pre_election: true
I20260811 07:09:30.074544 27851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } }
I20260811 07:09:30.074594 27589 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c62b7c927a54444a32d19bb1028f69d in term 1. Re-sending same reply.
I20260811 07:09:30.074987 27851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c62b7c927a54444a32d19bb1028f69d. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.075060 27851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:30.075431 27880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:30.075686 27880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:30.077858 27719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [term 1 FOLLOWER]: Refusing update from remote peer 3c62b7c927a54444a32d19bb1028f69d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:30.078243 27851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:30.079555 27782 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c62b7c927a54444a32d19bb1028f69d. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.079668 27782 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:09:30.080093 27589 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [term 1 FOLLOWER]: Refusing update from remote peer 3c62b7c927a54444a32d19bb1028f69d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:30.080406 27851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [LEADER]: Connected to new peer: Peer: permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:30.081629 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c62b7c927a54444a32d19bb1028f69d. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.081724 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:09:30.082787 27851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.082865 27851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:30.083304 27782 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.083387 27782 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:09:30.083616 27881 mvcc.cc:205] Tried to move back new op lower bound from 7317226168636473344 to 7317226168626085888. Current Snapshot: MvccSnapshot[applied={T|T < 7317226168636473344}]
I20260811 07:09:30.084203 27882 mvcc.cc:205] Tried to move back new op lower bound from 7317226168636473344 to 7317226168626085888. Current Snapshot: MvccSnapshot[applied={T|T < 7317226168636473344}]
I20260811 07:09:30.084770 27880 catalog_manager.cc:1347] Generated new cluster ID: a3814360a70b4ac0a4afc84fca8e67bc
I20260811 07:09:30.084820 27880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:30.089666 27876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.089759 27876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:30.090209 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c62b7c927a54444a32d19bb1028f69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c62b7c927a54444a32d19bb1028f69d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32839 } } peers { permanent_uuid: "073a6e30a339471ebbe9b03920fc415f" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36665 } } peers { permanent_uuid: "abf6a03526eb429fb4f48f000eda654c" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38791 } } }
I20260811 07:09:30.090306 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:09:30.090565 27885 mvcc.cc:205] Tried to move back new op lower bound from 7317226168636473344 to 7317226168626085888. Current Snapshot: MvccSnapshot[applied={T|T < 7317226168636473344}]
I20260811 07:09:30.099977 27880 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:09:30.100745 27880 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:30.104826 27880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c62b7c927a54444a32d19bb1028f69d: Generated new TSK 0
I20260811 07:09:30.105011 27880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:09:30.938359 27805 catalog_manager.cc:1259] Loaded cluster ID: a3814360a70b4ac0a4afc84fca8e67bc
I20260811 07:09:30.938506 27805 catalog_manager.cc:1552] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: loading cluster ID for follower catalog manager: success
I20260811 07:09:30.940194 27805 catalog_manager.cc:1574] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: acquiring CA information for follower catalog manager: success
I20260811 07:09:30.940744 27805 catalog_manager.cc:1602] T 00000000000000000000000000000000 P abf6a03526eb429fb4f48f000eda654c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:09:30.967295 27846 catalog_manager.cc:1259] Loaded cluster ID: a3814360a70b4ac0a4afc84fca8e67bc
I20260811 07:09:30.967518 27846 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: loading cluster ID for follower catalog manager: success
I20260811 07:09:30.969916 27846 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: acquiring CA information for follower catalog manager: success
I20260811 07:09:30.970901 27846 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 073a6e30a339471ebbe9b03920fc415f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:09:31.168717 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 27377
I20260811 07:09:31.214473 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 27518
I20260811 07:09:31.267931 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 27650
2026-08-11T07:09:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3323 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:09:31 dist-test-slave-2d19 krb5kdc[27910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:09:31 dist-test-slave-2d19 krb5kdc[27910](info): set up 2 sockets
Aug 11 07:09:31 dist-test-slave-2d19 krb5kdc[27910](info): commencing operation
krb5kdc: starting...
W20260811 07:09:32.569403 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[27910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:09:32.696099 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34371
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:32.780114 27930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:32.780412 27930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:32.780460 27930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:32.781790 27930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:32.781863 27930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:32.781896 27930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:32.781929 27930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:32.781957 27930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:32.783479 27930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:32.783768 27930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:32.784008 27930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:32.788991 27943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:32.789048 27942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:32.789171 27930 server_base.cc:1034] running on GCE node
W20260811 07:09:32.788992 27945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:32.789572 27930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:32.789880 27930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:32.791051 27930 hybrid_clock.cc:648] HybridClock initialized: now 1786432172791030 us; error 41 us; skew 500 ppm
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[27910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432172, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:32.798460 27930 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:32.798951 27930 webserver.cc:460] Webserver started at http://127.3.236.190:38473/ using document root <none> and password file <none>
I20260811 07:09:32.799196 27930 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:32.799268 27930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:32.799424 27930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:32.800647 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "61627b10677643a08a25e2ed63827a7e"
format_stamp: "Formatted at 2026-08-11 07:09:32 on dist-test-slave-2d19"
server_key: "af9ec723137e2d89a9a061c3c0eab540"
server_key_iv: "4e1e1e6dec12fffe309f1a05e75ad22c"
server_key_version: "encryptionkey@0"
I20260811 07:09:32.801019 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "61627b10677643a08a25e2ed63827a7e"
format_stamp: "Formatted at 2026-08-11 07:09:32 on dist-test-slave-2d19"
server_key: "af9ec723137e2d89a9a061c3c0eab540"
server_key_iv: "4e1e1e6dec12fffe309f1a05e75ad22c"
server_key_version: "encryptionkey@0"
I20260811 07:09:32.802371 27930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:09:32.803110 27962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:32.803279 27930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:09:32.803393 27930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "61627b10677643a08a25e2ed63827a7e"
format_stamp: "Formatted at 2026-08-11 07:09:32 on dist-test-slave-2d19"
server_key: "af9ec723137e2d89a9a061c3c0eab540"
server_key_iv: "4e1e1e6dec12fffe309f1a05e75ad22c"
server_key_version: "encryptionkey@0"
I20260811 07:09:32.803468 27930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:32.820508 27930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:32.823024 27930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:32.823194 27930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:32.827491 27930 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:34371
I20260811 07:09:32.827554 28056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:34371 every 8 connection(s)
I20260811 07:09:32.827922 27930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:32.828858 28058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:32.831140 28058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: Bootstrap starting.
I20260811 07:09:32.831391 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 27930
I20260811 07:09:32.831492 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:09:32.831708 4018 external_mini_cluster.cc:1428] Setting key 85b4ed09395407a3838a4be9eac09f6a
I20260811 07:09:32.831833 28058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:32.832211 28058 log.cc:824] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:32.832979 28058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: No bootstrap required, opened a new log
I20260811 07:09:32.835366 28058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } }
I20260811 07:09:32.835547 28058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:32.835589 28058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61627b10677643a08a25e2ed63827a7e, State: Initialized, Role: FOLLOWER
I20260811 07:09:32.835716 28058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } }
I20260811 07:09:32.835790 28058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:32.835819 28058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:32.835863 28058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:32.836578 28058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } }
I20260811 07:09:32.836673 28058 leader_election.cc:302] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61627b10677643a08a25e2ed63827a7e; no voters:
I20260811 07:09:32.836907 28058 leader_election.cc:288] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:09:32.836934 28065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:32.837056 28065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [term 1 LEADER]: Becoming Leader. State: Replica: 61627b10677643a08a25e2ed63827a7e, State: Running, Role: LEADER
I20260811 07:09:32.837145 28065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } }
I20260811 07:09:32.837194 28058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:32.837457 28065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61627b10677643a08a25e2ed63827a7e. Latest consensus state: current_term: 1 leader_uuid: "61627b10677643a08a25e2ed63827a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } } }
I20260811 07:09:32.837568 28065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:32.837455 28067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61627b10677643a08a25e2ed63827a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61627b10677643a08a25e2ed63827a7e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 34371 } } }
I20260811 07:09:32.838179 28067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e [sys.catalog]: This master's current role is: LEADER
W20260811 07:09:32.839109 28091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[27910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
W20260811 07:09:32.839175 28091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:09:32.839233 28072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:32.839418 28072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:32.840813 28072 catalog_manager.cc:1347] Generated new cluster ID: c76708e18dec42d48ca2ac41a2f32f68
I20260811 07:09:32.840869 28072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:32.843067 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:32.832305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47058 (local address 127.3.236.190:34371)
0811 07:09:32.832481 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:32.832491 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:32.832526 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:09:32.833104 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:32.833113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:32.833172 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:32.833286 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:32.834133 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:32.835141 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:32.836199 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:32.836718 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:32.836837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:32.839730 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:32.839734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:32.839734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:32.839746 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:32.841590 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:32.841949 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:32.841953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:32.841953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:32.842046 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:32.842275 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:32.842276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:32.842277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:32.842514 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:32.842670 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:32.842768 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 07:09:32.842900 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260811 07:09:32.844054 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:09:32.847838 28072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:09:32.848410 28072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:32.857621 28072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61627b10677643a08a25e2ed63827a7e: Generated new TSK 0
I20260811 07:09:32.857818 28072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:09:32.898017 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 27930
2026-08-11T07:09:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1626 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[28100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[28100](info): set up 2 sockets
Aug 11 07:09:32 dist-test-slave-2d19 krb5kdc[28100](info): commencing operation
krb5kdc: starting...
W20260811 07:09:34.195655 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:09:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:09:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:09:34.323336 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:34.407225 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:34.407473 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:34.407519 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:34.408691 28123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:34.408761 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:34.408787 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:34.408810 28123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:34.408838 28123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:34.410290 28123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:43623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:34.410567 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:34.410794 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:34.415902 28129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.415868 28132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:34.416087 28123 server_base.cc:1034] running on GCE node
W20260811 07:09:34.415886 28130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:34.416465 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:34.416766 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:34.417913 28123 hybrid_clock.cc:648] HybridClock initialized: now 1786432174417892 us; error 48 us; skew 500 ppm
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:34.423926 28123 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:34.424459 28123 webserver.cc:460] Webserver started at http://127.3.236.190:44441/ using document root <none> and password file <none>
I20260811 07:09:34.424695 28123 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:34.424765 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:34.424926 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:34.426129 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "6f499a2da8b54a70acf5e1714ba964b8"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "79b72ec0eaf355b7f1a5692617c20a99"
server_key_iv: "9bbd7e9bb59d22bd8a40f88c5f7d5506"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.426532 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "6f499a2da8b54a70acf5e1714ba964b8"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "79b72ec0eaf355b7f1a5692617c20a99"
server_key_iv: "9bbd7e9bb59d22bd8a40f88c5f7d5506"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.428021 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:09:34.428970 28145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.429168 28123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:34.429287 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "6f499a2da8b54a70acf5e1714ba964b8"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "79b72ec0eaf355b7f1a5692617c20a99"
server_key_iv: "9bbd7e9bb59d22bd8a40f88c5f7d5506"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.429368 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:34.439436 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:34.441566 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:34.441731 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:34.446309 28123 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:43623
I20260811 07:09:34.446357 28236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:43623 every 8 connection(s)
I20260811 07:09:34.446753 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:34.447831 28239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:34.448647 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 28123
I20260811 07:09:34.448774 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:09:34.448994 4018 external_mini_cluster.cc:1428] Setting key 539d04eac0d97f9ddb8f430c3de820b3
I20260811 07:09:34.451200 28239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap starting.
I20260811 07:09:34.451840 28239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:34.452198 28239 log.cc:824] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:34.453025 28239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: No bootstrap required, opened a new log
I20260811 07:09:34.455612 28239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:34.455752 28239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:34.455775 28239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Initialized, Role: FOLLOWER
I20260811 07:09:34.455914 28239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:34.455988 28239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:34.456033 28239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:34.456084 28239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:34.456856 28239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:34.456991 28239 leader_election.cc:302] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f499a2da8b54a70acf5e1714ba964b8; no voters:
I20260811 07:09:34.457160 28239 leader_election.cc:288] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:09:34.457283 28246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:34.457448 28239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:34.457466 28246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Running, Role: LEADER
I20260811 07:09:34.458173 28246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:34.458861 28247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:34.458959 28247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:34.459226 28248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f499a2da8b54a70acf5e1714ba964b8. Latest consensus state: current_term: 1 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:34.459419 28248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
W20260811 07:09:34.459429 28269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:09:34.459512 28269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:09:34.459671 28270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:34.459908 28270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:34.461030 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.449552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.3.236.190:43623)
0811 07:09:34.449740 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:34.449747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:34.449774 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:09:34.450190 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:34.450196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:34.450242 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:34.450358 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:34.451188 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.452232 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.453302 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.453705 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.453790 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.456687 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:34.456691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:34.456691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:34.456702 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:34.458744 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.459332 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.459335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.459336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.459452 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.459712 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.459713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.459713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.460046 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:34.460655 (+ 609us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:34.460714 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 07:09:34.460816 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260811 07:09:34.461583 28270 catalog_manager.cc:1347] Generated new cluster ID: f34608ec5d2547e4999f04cbaaf580d0
I20260811 07:09:34.461673 28270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:34.481259 28270 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:09:34.482031 28270 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:34.486333 28270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Generated new TSK 0
I20260811 07:09:34.486531 28270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:09:34.519467 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:34.605104 28281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:34.605350 28281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:34.605389 28281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:34.606520 28281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:34.606586 28281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:34.606630 28281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:34.608106 28281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:34.608407 28281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:34.608659 28281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:34.613777 28291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.613793 28290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.613793 28294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:34.614089 28281 server_base.cc:1034] running on GCE node
I20260811 07:09:34.614301 28281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:34.614579 28281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:34.615774 28281 hybrid_clock.cc:648] HybridClock initialized: now 1786432174615763 us; error 31 us; skew 500 ppm
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:34.618404 28281 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:34.618866 28281 webserver.cc:460] Webserver started at http://127.3.236.129:39359/ using document root <none> and password file <none>
I20260811 07:09:34.619071 28281 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:34.619113 28281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:34.619202 28281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:34.620190 28281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "81158b2eaee74b64adf77759f40c435c"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "fae99dcc5a6c9aceb8134292627922d3"
server_key_iv: "1c334365fea3d5f071e12a48ebfd3146"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.620572 28281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "81158b2eaee74b64adf77759f40c435c"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "fae99dcc5a6c9aceb8134292627922d3"
server_key_iv: "1c334365fea3d5f071e12a48ebfd3146"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.621855 28281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:34.622622 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.622781 28281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:09:34.622893 28281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "81158b2eaee74b64adf77759f40c435c"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "fae99dcc5a6c9aceb8134292627922d3"
server_key_iv: "1c334365fea3d5f071e12a48ebfd3146"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.622972 28281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:34.641346 28281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:34.643304 28281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:34.643466 28281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:34.643754 28281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:34.644119 28281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:34.644176 28281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.644217 28281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:34.644295 28281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.650520 28281 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:36501
I20260811 07:09:34.650630 28495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:36501 every 8 connection(s)
I20260811 07:09:34.650945 28281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:09:34.656333 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 28281
I20260811 07:09:34.656436 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:09:34.656661 4018 external_mini_cluster.cc:1428] Setting key d0c3b7e67046b0e4923968b8485308f9
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:34.661237 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.651764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:48785 (local address 127.3.236.190:43623)
0811 07:09:34.651834 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:34.651837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:34.652109 (+ 272us) server_negotiation.cc:407] Connection header received
0811 07:09:34.652623 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:34.652624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:34.652634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:34.652686 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:34.653632 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.654133 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.655109 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.655307 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.655358 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.658031 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:34.658032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:34.658032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:34.658039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:34.659739 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.660181 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.660182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.660182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.660290 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.660544 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.660546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.660546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.660668 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:34.660748 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:34.661047 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 07:09:34.661143 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:09:34.661432 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.651743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:48785)
0811 07:09:34.651975 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:34.652015 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:34.652535 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:34.652749 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:34.652753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:34.653350 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:34.653553 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.653557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.654221 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.654222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.655035 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.655036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.655382 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.655382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.655487 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.656066 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:34.656075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:34.657921 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:34.659830 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.659839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.659841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.660095 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.660353 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.660354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.660354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.660472 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.660739 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:34.660747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:34.660895 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 07:09:34.661024 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 07:09:34.661190 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260811 07:09:34.661947 28498 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:34.662062 28498 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:34.662282 28498 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:34.662904 28177 ts_manager.cc:194] Registered new tserver with Master: 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501)
I20260811 07:09:34.663558 28177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:48785
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:09:34.686949 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:34.781004 28509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:34.781229 28509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:34.781255 28509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:34.782564 28509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:34.782620 28509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:34.782650 28509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:34.784400 28509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:34.784673 28509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:34.784945 28509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:34.790843 28516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.790849 28515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.790849 28518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:34.791180 28509 server_base.cc:1034] running on GCE node
I20260811 07:09:34.791411 28509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:34.791659 28509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:34.792825 28509 hybrid_clock.cc:648] HybridClock initialized: now 1786432174792818 us; error 36 us; skew 500 ppm
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:34.795389 28509 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:34.795933 28509 webserver.cc:460] Webserver started at http://127.3.236.130:41783/ using document root <none> and password file <none>
I20260811 07:09:34.796191 28509 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:34.796262 28509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:34.796429 28509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:34.797489 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "aeac93dfbf6c4d939482238c243f5bc2"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "556cf77210fd2092bed90dd7a7c404fc"
server_key_iv: "c7ca512f6068c1a3208d0ff9cc50a6b1"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.797857 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "aeac93dfbf6c4d939482238c243f5bc2"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "556cf77210fd2092bed90dd7a7c404fc"
server_key_iv: "c7ca512f6068c1a3208d0ff9cc50a6b1"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.799219 28509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:09:34.799994 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.800169 28509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:34.800290 28509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "aeac93dfbf6c4d939482238c243f5bc2"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "556cf77210fd2092bed90dd7a7c404fc"
server_key_iv: "c7ca512f6068c1a3208d0ff9cc50a6b1"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.800379 28509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:34.820058 28509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:34.821988 28509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:34.822157 28509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:34.822391 28509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:34.822715 28509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:34.822772 28509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.822813 28509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:34.822841 28509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.829489 28509 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:38805
I20260811 07:09:34.829558 28729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:38805 every 8 connection(s)
I20260811 07:09:34.829857 28509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:09:34.833292 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 28509
I20260811 07:09:34.833407 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:09:34.833628 4018 external_mini_cluster.cc:1428] Setting key 7f46dd583ad70ab894f327fd8dee2ed6
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:34.840848 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.830581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:56895 (local address 127.3.236.190:43623)
0811 07:09:34.830628 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:34.830630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:34.830855 (+ 225us) server_negotiation.cc:407] Connection header received
0811 07:09:34.831320 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:34.831321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:34.831333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:34.831406 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:34.832422 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.833010 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.834410 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.834625 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.834681 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.837620 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:34.837622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:34.837622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:34.837629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:34.839257 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.839660 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.839661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.839662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.839728 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.839964 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.839965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.839966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.840126 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:34.840203 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:34.840605 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 07:09:34.840706 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 07:09:34.840868 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.830556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:56895)
0811 07:09:34.830708 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:34.830744 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:34.831221 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:34.831459 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:34.831462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:34.832109 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:34.832322 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.832327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.833114 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.833116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.834313 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.834316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.834710 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.834712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.834864 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.835379 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:34.835389 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:34.837520 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:34.839366 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.839373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.839375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.839600 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.839801 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.839801 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.839802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.839899 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.840223 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:34.840230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:34.840444 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 07:09:34.840554 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 07:09:34.840692 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260811 07:09:34.841454 28730 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:34.841568 28730 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:34.841787 28730 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:34.842195 28177 ts_manager.cc:194] Registered new tserver with Master: aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
I20260811 07:09:34.842621 28177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:56895
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:09:34.861346 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:34.948977 28740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:34.949236 28740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:34.949277 28740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:34.950435 28740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:34.950506 28740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:34.950547 28740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:34.952040 28740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:34.952350 28740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:34.952610 28740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:34.957981 28758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.958071 28755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:34.958041 28752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:34.958271 28740 server_base.cc:1034] running on GCE node
I20260811 07:09:34.958457 28740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:34.958690 28740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:34.959846 28740 hybrid_clock.cc:648] HybridClock initialized: now 1786432174959833 us; error 28 us; skew 500 ppm
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:34.962453 28740 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:34.962917 28740 webserver.cc:460] Webserver started at http://127.3.236.131:45231/ using document root <none> and password file <none>
I20260811 07:09:34.963107 28740 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:34.963150 28740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:34.963238 28740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:34.964296 28740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "f6cbb6676f09414d8e4253a70302dcfe"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "986649d2973d16e9e857441a3111bb67"
server_key_iv: "22355f40de96d5e143b2497b8c2a1d51"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.964645 28740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "f6cbb6676f09414d8e4253a70302dcfe"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "986649d2973d16e9e857441a3111bb67"
server_key_iv: "22355f40de96d5e143b2497b8c2a1d51"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.965992 28740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:34.966723 28768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.966907 28740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:09:34.967007 28740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f6cbb6676f09414d8e4253a70302dcfe"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "986649d2973d16e9e857441a3111bb67"
server_key_iv: "22355f40de96d5e143b2497b8c2a1d51"
server_key_version: "encryptionkey@0"
I20260811 07:09:34.967065 28740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:34.976824 28740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:34.979355 28740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:34.979482 28740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:34.979682 28740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:34.979971 28740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:34.980008 28740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.980032 28740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:34.980046 28740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:34.987063 28740 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35117
I20260811 07:09:34.987119 28954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35117 every 8 connection(s)
I20260811 07:09:34.987471 28740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:09:34.988458 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 28740
I20260811 07:09:34.988566 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:09:34.988763 4018 external_mini_cluster.cc:1428] Setting key b24c63f8bd173cc3c27d6e301b3b914d
Aug 11 07:09:34 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432174, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:34.997150 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.988290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:60855 (local address 127.3.236.190:43623)
0811 07:09:34.988347 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:34.988349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:34.988562 (+ 213us) server_negotiation.cc:407] Connection header received
0811 07:09:34.989259 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:34.989260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:34.989272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:34.989329 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:34.990347 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.990850 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.991899 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.992101 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.992180 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.994387 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:34.994389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:34.994390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:34.994398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:34.995669 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.996089 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.996090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.996090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.996178 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:34.996484 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:34.996486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:34.996486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:34.996606 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:34.996684 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:34.996976 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 07:09:34.997060 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:09:34.997334 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:34.988215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:60855)
0811 07:09:34.988416 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:34.988459 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:34.989145 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:34.989405 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:34.989409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:34.990039 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:34.990244 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.990249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.990974 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.990976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.991791 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:34.991793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:34.992191 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:34.992192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:34.992363 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:34.992854 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:34.992862 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:34.994287 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:34.995759 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.995767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.995770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.996005 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.996296 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:34.996296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:34.996297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:34.996409 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:34.996689 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:34.996697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:34.996833 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 07:09:34.996940 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 07:09:34.997098 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260811 07:09:34.997821 28955 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:34.997906 28955 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:34.998154 28955 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:34.998574 28177 ts_manager.cc:194] Registered new tserver with Master: f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117)
I20260811 07:09:34.999023 28177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:60855
I20260811 07:09:35.000653 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:09:35.007890 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.001893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.3.236.190:43623)
0811 07:09:35.001958 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.001960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.002063 (+ 103us) server_negotiation.cc:407] Connection header received
0811 07:09:35.002104 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.002104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.002113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.002163 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:35.002696 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.003229 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.003903 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.004097 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.004151 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.005369 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:35.005370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:35.005371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:35.005376 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:35.006818 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.007208 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.007209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.007209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.007261 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.007469 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.007469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.007469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.007602 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:35.007677 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.007770 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.007827 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:09:35.009487 28177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:09:35.011170 28177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:09:35.018069 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.013715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.0.0.1:60314)
0811 07:09:35.013948 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.014001 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.014073 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.014365 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.014368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.014370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.014462 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.014465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.015843 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.015844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.016911 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.016914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.017769 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.017770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.017850 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.017863 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.017935 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.017988 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":70,"thread_start_us":61,"threads_started":1}
I20260811 07:09:35.018134 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.013828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.3.236.129:36501)
0811 07:09:35.014034 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.014039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.014051 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:09:35.014176 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.014179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.014198 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.014274 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.014541 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.015718 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.017036 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.017599 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.017674 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.017733 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.017981 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.018040 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260811 07:09:35.018595 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.013782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35117 (local address 127.0.0.1:52814)
0811 07:09:35.014181 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.014190 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.014232 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.014867 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.014868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.014870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.014946 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.014948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.016122 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.016122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.017362 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.017365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.018344 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.018345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.018414 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.018418 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.018467 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.018511 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":110,"threads_started":1}
I20260811 07:09:35.018779 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.014071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.3.236.130:38805)
0811 07:09:35.014304 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.014309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.014462 (+ 153us) server_negotiation.cc:407] Connection header received
0811 07:09:35.014624 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.014626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.014643 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.014717 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.014935 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.015747 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.017676 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.018384 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.018493 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.018552 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.018654 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.018694 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
I20260811 07:09:35.018885 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.014105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.3.236.131:35117)
0811 07:09:35.014320 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.014324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.014338 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:09:35.014387 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.014390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.014407 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.014772 (+ 365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.015042 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.016028 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.017455 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.018180 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.018257 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.018312 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.018747 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.018799 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":99,"threads_started":1}
I20260811 07:09:35.019004 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.013782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:38805 (local address 127.0.0.1:45498)
0811 07:09:35.014359 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.014371 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.014495 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.014800 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.014801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.014803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.014848 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.014849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.015890 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.015891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.017580 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.017582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.018499 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.018500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.018570 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.018575 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.018638 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.018680 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":38,"thread_start_us":101,"threads_started":1}
I20260811 07:09:35.019274 28392 tablet_service.cc:1467] Processing CreateTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 (DEFAULT_TABLE table=test-table [id=83e0463e6e10492286fce4855bd07e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.019639 28392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea9ed2b12e47a6a27f2a782628b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.019641 28853 tablet_service.cc:1467] Processing CreateTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 (DEFAULT_TABLE table=test-table [id=83e0463e6e10492286fce4855bd07e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.019754 28610 tablet_service.cc:1467] Processing CreateTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 (DEFAULT_TABLE table=test-table [id=83e0463e6e10492286fce4855bd07e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.020021 28853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea9ed2b12e47a6a27f2a782628b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.020059 28610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea9ed2b12e47a6a27f2a782628b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.022295 28976 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Bootstrap starting.
I20260811 07:09:35.023097 28976 tablet_bootstrap.cc:654] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.023660 28980 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Bootstrap starting.
I20260811 07:09:35.023870 28976 log.cc:824] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.024327 28979 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Bootstrap starting.
I20260811 07:09:35.024466 28980 tablet_bootstrap.cc:654] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.024776 28976 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: No bootstrap required, opened a new log
I20260811 07:09:35.024873 28976 ts_tablet_manager.cc:1397] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:09:35.025071 28979 tablet_bootstrap.cc:654] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.025067 28980 log.cc:824] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.025738 28979 log.cc:824] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.025872 28980 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: No bootstrap required, opened a new log
I20260811 07:09:35.025947 28980 ts_tablet_manager.cc:1397] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:09:35.026650 28979 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: No bootstrap required, opened a new log
I20260811 07:09:35.026732 28979 ts_tablet_manager.cc:1397] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:09:35.027588 28980 raft_consensus.cc:348] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.027784 28980 raft_consensus.cc:374] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.027833 28980 raft_consensus.cc:729] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeac93dfbf6c4d939482238c243f5bc2, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.027798 28976 raft_consensus.cc:348] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.028007 28976 raft_consensus.cc:374] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.027956 28980 consensus_queue.cc:260] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.028054 28976 raft_consensus.cc:729] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81158b2eaee74b64adf77759f40c435c, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.028204 28976 consensus_queue.cc:260] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.028306 28980 ts_tablet_manager.cc:1428] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:09:35.028440 28730 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.028537 28976 ts_tablet_manager.cc:1428] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 07:09:35.028610 28498 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.029049 28979 raft_consensus.cc:348] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.029242 28979 raft_consensus.cc:374] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.029295 28979 raft_consensus.cc:729] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cbb6676f09414d8e4253a70302dcfe, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.029459 28979 consensus_queue.cc:260] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.029795 28979 ts_tablet_manager.cc:1428] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:09:35.029840 28955 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.040225 28991 raft_consensus.cc:482] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:35.040370 28991 raft_consensus.cc:504] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.040723 28991 leader_election.cc:288] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501), f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117)
I20260811 07:09:35.043761 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.040966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:57795 (local address 127.3.236.129:36501)
0811 07:09:35.041011 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.041013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.041024 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:09:35.041112 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.041114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.041125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.041178 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.041352 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.041864 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.042855 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.043394 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.043458 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.043468 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.043639 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.043688 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:09:35.043783 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.040862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.3.236.130:57795)
0811 07:09:35.040924 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.040937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.041030 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.041193 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.041195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.041197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.041276 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.041278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.041991 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.041992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.042761 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.042764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.043473 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.043474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.043566 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.043570 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.043627 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.043680 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":20}
I20260811 07:09:35.043807 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.040974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:36823 (local address 127.3.236.131:35117)
0811 07:09:35.041042 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.041045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.041181 (+ 136us) server_negotiation.cc:407] Connection header received
0811 07:09:35.041211 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.041212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.041221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.041282 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.041465 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.041927 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.042884 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.043476 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.043542 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.043551 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.043703 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.043744 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:09:35.043918 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.040922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35117 (local address 127.3.236.130:36823)
0811 07:09:35.041089 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.041100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.041163 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.041293 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.041294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.041295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.041383 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.041384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.042014 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.042015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.042772 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.042774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.043542 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.043543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.043618 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.043623 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.043683 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.043722 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":4,"thread_start_us":109,"threads_started":1}
I20260811 07:09:35.044205 28881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea9ed2b12e47a6a27f2a782628b9e5" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cbb6676f09414d8e4253a70302dcfe" is_pre_election: true
I20260811 07:09:35.044193 28426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea9ed2b12e47a6a27f2a782628b9e5" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81158b2eaee74b64adf77759f40c435c" is_pre_election: true
I20260811 07:09:35.044363 28881 raft_consensus.cc:2454] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 0.
I20260811 07:09:35.044363 28426 raft_consensus.cc:2454] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 0.
I20260811 07:09:35.044622 28529 leader_election.cc:302] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe; no voters:
I20260811 07:09:35.044775 28991 raft_consensus.cc:2781] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:09:35.044831 28991 raft_consensus.cc:482] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:09:35.044883 28991 raft_consensus.cc:3037] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.045641 28991 raft_consensus.cc:504] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.045789 28991 leader_election.cc:288] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501), f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117)
I20260811 07:09:35.045975 28881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea9ed2b12e47a6a27f2a782628b9e5" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cbb6676f09414d8e4253a70302dcfe"
I20260811 07:09:35.046060 28881 raft_consensus.cc:3037] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.046769 28881 raft_consensus.cc:2454] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 1.
I20260811 07:09:35.046903 28426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea9ed2b12e47a6a27f2a782628b9e5" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81158b2eaee74b64adf77759f40c435c"
I20260811 07:09:35.046974 28529 leader_election.cc:302] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe; no voters:
I20260811 07:09:35.046994 28426 raft_consensus.cc:3037] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.047053 28991 raft_consensus.cc:2781] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:35.047215 28991 raft_consensus.cc:686] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 LEADER]: Becoming Leader. State: Replica: aeac93dfbf6c4d939482238c243f5bc2, State: Running, Role: LEADER
I20260811 07:09:35.047317 28991 consensus_queue.cc:237] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.047971 28426 raft_consensus.cc:2454] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 1.
I20260811 07:09:35.048229 28175 catalog_manager.cc:5495] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "aeac93dfbf6c4d939482238c243f5bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } health_report { overall_health: UNKNOWN } } }
I20260811 07:09:35.055562 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.053073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.3.236.130:38805)
0811 07:09:35.053121 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.053123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.053149 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:09:35.053182 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.053183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.053201 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.053285 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:09:35.053433 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.053907 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.054737 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.054951 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.055030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.055209 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:09:35.055315 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:09:35.055412 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.055431 (+ 19us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.055459 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:09:35.057474 28426 raft_consensus.cc:1261] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Refusing update from remote peer aeac93dfbf6c4d939482238c243f5bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:35.057485 28881 raft_consensus.cc:1261] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Refusing update from remote peer aeac93dfbf6c4d939482238c243f5bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:35.057876 28991 consensus_queue.cc:1035] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:35.058037 28998 consensus_queue.cc:1035] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:35.060642 29003 mvcc.cc:205] Tried to move back new op lower bound from 7317226189032493056 to 7317226188994494464. Current Snapshot: MvccSnapshot[applied={T|T < 7317226189032493056}]
I20260811 07:09:35.060711 29002 mvcc.cc:205] Tried to move back new op lower bound from 7317226189032493056 to 7317226188994494464. Current Snapshot: MvccSnapshot[applied={T|T < 7317226189032493056}]
I20260811 07:09:35.061208 29008 mvcc.cc:205] Tried to move back new op lower bound from 7317226189032493056 to 7317226188994494464. Current Snapshot: MvccSnapshot[applied={T|T < 7317226189032493056}]
I20260811 07:09:35.062881 28175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:35.062987 28175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:35.063787 28175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Sending DeleteTablet for 3 replicas of tablet ceea9ed2b12e47a6a27f2a782628b9e5
I20260811 07:09:35.064182 28392 tablet_service.cc:1514] Processing DeleteTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:60314
I20260811 07:09:35.064234 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 28281
I20260811 07:09:35.064345 28610 tablet_service.cc:1514] Processing DeleteTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45498
I20260811 07:09:35.064834 28853 tablet_service.cc:1514] Processing DeleteTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:52814
I20260811 07:09:35.065070 29018 tablet_replica.cc:330] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: stopping tablet replica
I20260811 07:09:35.065204 29018 raft_consensus.cc:2229] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:35.065343 29018 raft_consensus.cc:2258] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:35.072374 29018 ts_tablet_manager.cc:1905] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:35.074404 29018 ts_tablet_manager.cc:1918] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:35.074528 29018 log.cc:1196] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/ceea9ed2b12e47a6a27f2a782628b9e5
I20260811 07:09:35.074821 29018 ts_tablet_manager.cc:1939] T ceea9ed2b12e47a6a27f2a782628b9e5 P f6cbb6676f09414d8e4253a70302dcfe: Deleting consensus metadata
I20260811 07:09:35.075294 28146 catalog_manager.cc:4841] TS f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): tablet ceea9ed2b12e47a6a27f2a782628b9e5 (table test-table [id=83e0463e6e10492286fce4855bd07e40]) successfully deleted
I20260811 07:09:35.083907 29017 tablet_replica.cc:330] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: stopping tablet replica
I20260811 07:09:35.084060 29017 raft_consensus.cc:2229] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:09:35.084218 29017 raft_consensus.cc:2258] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:35.084790 29017 ts_tablet_manager.cc:1905] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:35.086954 29017 ts_tablet_manager.cc:1918] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:35.087044 29017 log.cc:1196] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/ceea9ed2b12e47a6a27f2a782628b9e5
I20260811 07:09:35.087347 29017 ts_tablet_manager.cc:1939] T ceea9ed2b12e47a6a27f2a782628b9e5 P aeac93dfbf6c4d939482238c243f5bc2: Deleting consensus metadata
I20260811 07:09:35.087888 28147 catalog_manager.cc:4841] TS aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): tablet ceea9ed2b12e47a6a27f2a782628b9e5 (table test-table [id=83e0463e6e10492286fce4855bd07e40]) successfully deleted
W20260811 07:09:35.088887 28149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:36501 (error 108)
I20260811 07:09:35.089331 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 28509
I20260811 07:09:35.106428 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.105530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.0.0.1:60322)
0811 07:09:35.106323 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.106380 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:36501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":762}
W20260811 07:09:35.106587 28149 catalog_manager.cc:4584] TS 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ceea9ed2b12e47a6a27f2a782628b9e5: Network error: Client connection negotiation failed: client connection to 127.3.236.129:36501: connect: Connection refused (error 111)
I20260811 07:09:35.116062 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 28740
I20260811 07:09:35.127970 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 28123
I20260811 07:09:35.138615 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--webserver_interface=127.3.236.190
--webserver_port=44441
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:35.223913 29020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:35.224159 29020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:35.224197 29020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:35.225405 29020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:35.225472 29020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:35.225497 29020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:35.225514 29020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:35.225533 29020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:35.227046 29020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:43623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=44441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:35.227322 29020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:35.227550 29020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:35.240748 29033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.240742 29032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.240742 29041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:35.240959 29020 server_base.cc:1034] running on GCE node
I20260811 07:09:35.241305 29020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:35.241629 29020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:35.242776 29020 hybrid_clock.cc:648] HybridClock initialized: now 1786432175242757 us; error 38 us; skew 500 ppm
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:35.245535 29020 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:35.246081 29020 webserver.cc:460] Webserver started at http://127.3.236.190:44441/ using document root <none> and password file <none>
I20260811 07:09:35.246343 29020 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:35.246419 29020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:35.247820 29020 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:35.248586 29055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.248775 29020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:35.248874 29020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "6f499a2da8b54a70acf5e1714ba964b8"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "79b72ec0eaf355b7f1a5692617c20a99"
server_key_iv: "9bbd7e9bb59d22bd8a40f88c5f7d5506"
server_key_version: "encryptionkey@0"
I20260811 07:09:35.249310 29020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:35.263821 29020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:35.266201 29020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:35.266361 29020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:35.270740 29020 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:43623
I20260811 07:09:35.270803 29160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:43623 every 8 connection(s)
I20260811 07:09:35.271183 29020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:35.272970 29161 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:09:35.273581 29161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap starting.
I20260811 07:09:35.273907 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 29020
I20260811 07:09:35.274262 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:36501
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=39359
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:35.283340 29161 log.cc:824] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.285542 29161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:09:35.285872 29161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap complete.
I20260811 07:09:35.288527 29161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.288833 29161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.289017 29161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.289074 29161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:35.289147 29161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:35.289201 29161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:09:35.290239 29161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.290362 29161 leader_election.cc:302] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f499a2da8b54a70acf5e1714ba964b8; no voters:
I20260811 07:09:35.290591 29161 leader_election.cc:288] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:09:35.290627 29169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:09:35.290748 29169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 LEADER]: Becoming Leader. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Running, Role: LEADER
I20260811 07:09:35.290887 29161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:35.290942 29169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.291334 29173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f499a2da8b54a70acf5e1714ba964b8. Latest consensus state: current_term: 2 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:35.291429 29173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:35.291657 29172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:35.291738 29172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:35.291847 29179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:35.292904 29179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:35.293083 29179 catalog_manager.cc:1259] Loaded cluster ID: f34608ec5d2547e4999f04cbaaf580d0
I20260811 07:09:35.293326 29179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:09:35.293568 29179 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 07:09:35.296306 29179 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 07:09:35.298149 29179 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:35.298357 29179 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Loaded TSK: 0
I20260811 07:09:35.298575 29179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:09:35.371109 29166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:35.371346 29166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:35.371387 29166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:35.372609 29166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:35.372670 29166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:35.372715 29166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:35.374296 29166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:36501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=39359
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:35.374567 29166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:35.374816 29166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:35.380817 29215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.380836 29209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.380821 29210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:35.381167 29166 server_base.cc:1034] running on GCE node
I20260811 07:09:35.381367 29166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:35.381610 29166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:35.382784 29166 hybrid_clock.cc:648] HybridClock initialized: now 1786432175382760 us; error 39 us; skew 500 ppm
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:35.385391 29166 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:35.385864 29166 webserver.cc:460] Webserver started at http://127.3.236.129:39359/ using document root <none> and password file <none>
I20260811 07:09:35.386101 29166 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:35.386188 29166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:35.387740 29166 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:09:35.388515 29227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.388670 29166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:35.388764 29166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "81158b2eaee74b64adf77759f40c435c"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "fae99dcc5a6c9aceb8134292627922d3"
server_key_iv: "1c334365fea3d5f071e12a48ebfd3146"
server_key_version: "encryptionkey@0"
I20260811 07:09:35.389091 29166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:35.401464 29166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:35.403836 29166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:35.403980 29166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:35.404250 29166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:35.404791 29240 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:09:35.406332 29166 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:09:35.406391 29166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 07:09:35.406417 29166 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:09:35.407085 29166 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:09:35.407130 29166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 07:09:35.407186 29240 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Bootstrap starting.
I20260811 07:09:35.415261 29166 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:36501
I20260811 07:09:35.415373 29415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:36501 every 8 connection(s)
I20260811 07:09:35.415961 29166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:09:35.419273 29240 log.cc:824] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.420737 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 29166
I20260811 07:09:35.421067 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:38805
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=41783
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:35.423858 29240 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:09:35.424168 29240 tablet_bootstrap.cc:492] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Bootstrap complete.
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:35.425541 29240 ts_tablet_manager.cc:1397] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.007s
I20260811 07:09:35.427080 29240 raft_consensus.cc:348] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.427515 29240 raft_consensus.cc:729] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81158b2eaee74b64adf77759f40c435c, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.427745 29240 consensus_queue.cc:260] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } }
I20260811 07:09:35.428128 29240 ts_tablet_manager.cc:1428] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:09:35.429673 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.416709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:39117)
0811 07:09:35.417012 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.417056 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.417657 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.418365 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.418370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.418825 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:35.419104 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.419113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.420827 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.420828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.422032 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.422034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.422814 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.422815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.422955 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.423558 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:35.423568 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:35.425392 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:35.427846 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.427852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.427854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.428170 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.428521 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.428522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.428522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.428627 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.429032 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:35.429039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:35.429173 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.429276 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.429460 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":102,"threads_started":1}
I20260811 07:09:35.429704 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.416825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:39117 (local address 127.3.236.190:43623)
0811 07:09:35.417877 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.417886 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.417922 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:09:35.418063 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.418069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.418124 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.418269 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:35.419273 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.420690 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.422147 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.422700 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.423244 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.425530 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:35.425535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:35.425535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:35.425548 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:35.427735 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.428314 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.428317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.428317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.428424 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.428699 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.428700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.428700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.428934 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:35.429051 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.429349 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.429478 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":100,"threads_started":1}
I20260811 07:09:35.430627 29418 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:35.430727 29418 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:35.430907 29418 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:35.432089 29088 ts_manager.cc:194] Registered new tserver with Master: 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501)
I20260811 07:09:35.433286 29088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:39117
I20260811 07:09:35.434134 29418 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.439508 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.432759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.0.0.1:60328)
0811 07:09:35.436384 (+ 3625us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.436416 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.436473 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.436770 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.436773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.436775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.436833 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.436835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.437649 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.437650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.438453 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.438456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.439170 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.439171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.439257 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.439270 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.439372 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.439430 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3594,"thread_start_us":96,"threads_started":1}
I20260811 07:09:35.439509 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.433002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.3.236.129:36501)
0811 07:09:35.436619 (+ 3617us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.436624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.436634 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:09:35.436671 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.436673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.436688 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.436768 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.436891 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.437555 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.438565 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.439082 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.439169 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.439226 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.439392 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.439431 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3584,"thread_start_us":65,"threads_started":1}
I20260811 07:09:35.440207 29321 tablet_service.cc:1514] Processing DeleteTablet for tablet ceea9ed2b12e47a6a27f2a782628b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:60328
I20260811 07:09:35.440793 29436 tablet_replica.cc:330] stopping tablet replica
I20260811 07:09:35.440922 29436 raft_consensus.cc:2229] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:35.441001 29436 raft_consensus.cc:2258] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:35.441311 29436 ts_tablet_manager.cc:1905] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:35.443274 29436 ts_tablet_manager.cc:1918] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:35.443367 29436 log.cc:1196] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/ceea9ed2b12e47a6a27f2a782628b9e5
I20260811 07:09:35.443637 29436 ts_tablet_manager.cc:1939] T ceea9ed2b12e47a6a27f2a782628b9e5 P 81158b2eaee74b64adf77759f40c435c: Deleting consensus metadata
I20260811 07:09:35.444161 29062 catalog_manager.cc:4841] TS 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501): tablet ceea9ed2b12e47a6a27f2a782628b9e5 (table test-table [id=83e0463e6e10492286fce4855bd07e40]) successfully deleted
W20260811 07:09:35.515379 29426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:35.515575 29426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:35.515599 29426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:35.516780 29426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:35.516824 29426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:35.516851 29426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:35.518321 29426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:38805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=41783
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:35.518572 29426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:35.518815 29426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:35.524528 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.524561 29444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.524864 29450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:35.525053 29426 server_base.cc:1034] running on GCE node
I20260811 07:09:35.525249 29426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:35.525454 29426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:35.526621 29426 hybrid_clock.cc:648] HybridClock initialized: now 1786432175526604 us; error 32 us; skew 500 ppm
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:35.529402 29426 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:35.529898 29426 webserver.cc:460] Webserver started at http://127.3.236.130:41783/ using document root <none> and password file <none>
I20260811 07:09:35.530156 29426 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:35.530246 29426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:35.532140 29426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:09:35.532892 29466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.533078 29426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:09:35.533165 29426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "aeac93dfbf6c4d939482238c243f5bc2"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "556cf77210fd2092bed90dd7a7c404fc"
server_key_iv: "c7ca512f6068c1a3208d0ff9cc50a6b1"
server_key_version: "encryptionkey@0"
I20260811 07:09:35.533712 29426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:35.541873 29426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:35.544139 29426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:35.544337 29426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:35.544586 29426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:35.544898 29426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:35.544953 29426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.544993 29426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:35.545020 29426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.551769 29426 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:38805
I20260811 07:09:35.551815 29661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:38805 every 8 connection(s)
I20260811 07:09:35.552189 29426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:35.560743 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 29426
I20260811 07:09:35.561094 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:35117
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=45231
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:35.561510 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.552970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:53345 (local address 127.3.236.190:43623)
0811 07:09:35.553022 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.553024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.553287 (+ 263us) server_negotiation.cc:407] Connection header received
0811 07:09:35.553772 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.553773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.553788 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.553843 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:35.554769 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.555261 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.556245 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.556489 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.556545 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.558831 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:35.558832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:35.558833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:35.558840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:35.560070 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.560439 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.560440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.560441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.560508 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.560804 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.560805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.560805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.560967 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:35.561038 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.561369 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.561444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:09:35.561818 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.552992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:53345)
0811 07:09:35.553162 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.553198 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.553684 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.553897 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.553901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.554477 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:35.554685 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.554690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.555347 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.555348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.556169 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.556171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.556595 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.556595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.556704 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.557302 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:35.557311 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:35.558740 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:35.560144 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.560150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.560152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.560378 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.560576 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.560577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.560577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.560728 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.561052 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:35.561060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:35.561228 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.561348 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.561534 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260811 07:09:35.562469 29662 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:35.562585 29662 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:35.562788 29662 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:35.563277 29088 ts_manager.cc:194] Registered new tserver with Master: aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
I20260811 07:09:35.563657 29088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:53345
W20260811 07:09:35.652841 29668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:35.653143 29668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:35.653191 29668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:35.654490 29668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:35.654565 29668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:35.654615 29668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:35.656319 29668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:35117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=45231
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:35.656648 29668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:35.656929 29668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:35.662629 29678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.662637 29680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.662642 29683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:35.662669 29668 server_base.cc:1034] running on GCE node
I20260811 07:09:35.663126 29668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:35.663391 29668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:35.664569 29668 hybrid_clock.cc:648] HybridClock initialized: now 1786432175664563 us; error 39 us; skew 500 ppm
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:35.667368 29668 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:35.667855 29668 webserver.cc:460] Webserver started at http://127.3.236.131:45231/ using document root <none> and password file <none>
I20260811 07:09:35.668119 29668 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:35.668206 29668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:35.669657 29668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:35.670344 29693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.670562 29668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:35.670671 29668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f6cbb6676f09414d8e4253a70302dcfe"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "986649d2973d16e9e857441a3111bb67"
server_key_iv: "22355f40de96d5e143b2497b8c2a1d51"
server_key_version: "encryptionkey@0"
I20260811 07:09:35.671084 29668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:35.694911 29668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:35.697479 29668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:35.697657 29668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:35.697928 29668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:35.698271 29668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:35.698328 29668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.698370 29668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:35.698416 29668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.704823 29668 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35117
I20260811 07:09:35.704907 29877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35117 every 8 connection(s)
I20260811 07:09:35.705292 29668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:09:35.707932 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 29668
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:35.715729 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.708409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.3.236.190:43623)
0811 07:09:35.708536 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.708538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.708656 (+ 118us) server_negotiation.cc:407] Connection header received
0811 07:09:35.708688 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.708689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.708700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.708779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:35.709471 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.709912 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.710998 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.711265 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.711316 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.712609 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:35.712611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:35.712611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:35.712620 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:35.714289 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.714792 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.714793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.714793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.714848 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.715109 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.715111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.715111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.715238 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:35.715374 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.715489 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.715596 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260811 07:09:35.716790 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.706207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:36409 (local address 127.3.236.190:43623)
0811 07:09:35.706261 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.706263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.706526 (+ 263us) server_negotiation.cc:407] Connection header received
0811 07:09:35.707104 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.707105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.707118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.707175 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:35.708293 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.709003 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.710235 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.710411 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.710469 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.713236 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:35.713237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:35.713238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:35.713245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:35.715113 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.715600 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.715601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.715601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.715731 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:35.716071 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:35.716072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:35.716072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:35.716259 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:35.716355 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.716667 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.716731 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:09:35.717041 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.706166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:36409)
0811 07:09:35.706378 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.706418 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.706985 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.707245 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.707249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.707870 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:35.708135 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.708142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.709097 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.709099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.710147 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.710151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.710496 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.710497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.710641 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.711235 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:35.711244 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:35.713113 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:35.715173 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.715182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.715184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.715511 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.715801 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:35.715802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:35.715802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:35.715957 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:35.716343 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:35.716352 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:35.716511 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.716629 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.716763 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260811 07:09:35.717514 29878 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43623
I20260811 07:09:35.717597 29878 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:35.717803 29878 heartbeater.cc:511] Master 127.3.236.190:43623 requested a full tablet report, sending...
I20260811 07:09:35.718191 29088 ts_manager.cc:194] Registered new tserver with Master: f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117)
I20260811 07:09:35.718546 29088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:36409
I20260811 07:09:35.727514 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:09:35.730472 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.727831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.3.236.190:43623)
0811 07:09:35.727893 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.727895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.728013 (+ 118us) server_negotiation.cc:407] Connection header received
0811 07:09:35.728097 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.728098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.728143 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.728228 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:09:35.728393 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.728876 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.729701 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.729909 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.729986 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.730149 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:09:35.730251 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:09:35.730339 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.730371 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.730388 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:09:35.730780 29088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:09:35.732476 29088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:09:35.734754 29321 tablet_service.cc:1467] Processing CreateTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 (DEFAULT_TABLE table=test-table [id=21e04de56452414e9c01b3a02829c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.734990 29321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a52d6e3e2741f4b7ef3374c7357fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.736526 29240 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Bootstrap starting.
I20260811 07:09:35.736865 29240 tablet_bootstrap.cc:654] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.737474 29240 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: No bootstrap required, opened a new log
I20260811 07:09:35.737576 29240 ts_tablet_manager.cc:1397] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:35.737695 29240 raft_consensus.cc:348] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.737774 29240 raft_consensus.cc:374] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.737857 29240 raft_consensus.cc:729] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81158b2eaee74b64adf77759f40c435c, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.737902 29240 consensus_queue.cc:260] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.737999 29240 ts_tablet_manager.cc:1428] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.739054 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.734451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35117 (local address 127.0.0.1:52830)
0811 07:09:35.734500 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.734514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.734553 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.734960 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.734962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.734965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.735016 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.735017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.736083 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.736084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.737177 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.737183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.738860 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.738861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.738930 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.738933 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.738972 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.739006 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:09:35.739116 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.734558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.3.236.131:35117)
0811 07:09:35.734750 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.734755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.734766 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:09:35.734805 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.734807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.734823 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.734958 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.735095 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.735988 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.737276 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.738775 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.738852 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.738907 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.738994 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.739033 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260811 07:09:35.739809 29779 tablet_service.cc:1467] Processing CreateTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 (DEFAULT_TABLE table=test-table [id=21e04de56452414e9c01b3a02829c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.740187 29779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a52d6e3e2741f4b7ef3374c7357fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.740628 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.734473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:38805 (local address 127.0.0.1:45514)
0811 07:09:35.736579 (+ 2106us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.736591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.736716 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.736981 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.736982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.736985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.737024 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.737025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.738396 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.738397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.739456 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.739459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.740450 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.740451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.740500 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.740504 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.740548 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.740578 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2087,"thread_start_us":131,"threads_started":1}
I20260811 07:09:35.740710 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.734761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.3.236.130:38805)
0811 07:09:35.735032 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.735036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.736684 (+ 1648us) server_negotiation.cc:407] Connection header received
0811 07:09:35.736809 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.736817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.736829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.736916 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.737140 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.738283 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.739539 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.740357 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.740448 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.740505 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.740573 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.740623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":104,"threads_started":1}
I20260811 07:09:35.741326 29550 tablet_service.cc:1467] Processing CreateTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 (DEFAULT_TABLE table=test-table [id=21e04de56452414e9c01b3a02829c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:35.741631 29550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77a52d6e3e2741f4b7ef3374c7357fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:35.743336 29899 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Bootstrap starting.
I20260811 07:09:35.743924 29899 tablet_bootstrap.cc:654] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.744489 29899 log.cc:824] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.744597 29900 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Bootstrap starting.
I20260811 07:09:35.745162 29900 tablet_bootstrap.cc:654] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:35.745258 29899 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: No bootstrap required, opened a new log
I20260811 07:09:35.745345 29899 ts_tablet_manager.cc:1397] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:09:35.745690 29900 log.cc:824] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.746444 29900 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: No bootstrap required, opened a new log
I20260811 07:09:35.746546 29900 ts_tablet_manager.cc:1397] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:09:35.747229 29899 raft_consensus.cc:348] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.747391 29899 raft_consensus.cc:374] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.747449 29899 raft_consensus.cc:729] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cbb6676f09414d8e4253a70302dcfe, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.747568 29899 consensus_queue.cc:260] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.747861 29899 ts_tablet_manager.cc:1428] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 07:09:35.747910 29878 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.748216 29900 raft_consensus.cc:348] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.748436 29900 raft_consensus.cc:374] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:35.748481 29900 raft_consensus.cc:729] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeac93dfbf6c4d939482238c243f5bc2, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.748607 29900 consensus_queue.cc:260] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.748903 29900 ts_tablet_manager.cc:1428] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 07:09:35.748946 29662 heartbeater.cc:503] Master 127.3.236.190:43623 was elected leader, sending a full tablet report...
I20260811 07:09:35.777966 29906 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:35.778083 29906 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.778460 29906 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501)
I20260811 07:09:35.781508 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.778658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:58165 (local address 127.3.236.131:35117)
0811 07:09:35.778706 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.778709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.778720 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:09:35.778813 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.778816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.778831 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.778895 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.779075 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.779553 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.780748 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.781138 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.781225 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.781234 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.781387 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.781433 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:09:35.781513 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.778594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35117 (local address 127.3.236.130:58165)
0811 07:09:35.778651 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.778665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.778714 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.778916 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.778917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.778918 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.778994 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.778996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.779641 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.779642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.780655 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.780658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.781224 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.781225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.781308 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.781311 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.781379 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.781451 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":68}
I20260811 07:09:35.781617 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.778728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:44295 (local address 127.3.236.129:36501)
0811 07:09:35.778775 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.778777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.778882 (+ 105us) server_negotiation.cc:407] Connection header received
0811 07:09:35.778955 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.778956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.778966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.779044 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:35.779209 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.779667 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.780857 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.781235 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.781324 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.781333 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.781499 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.781540 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:09:35.781843 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.778610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.3.236.130:44295)
0811 07:09:35.778799 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.778809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:35.778898 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:35.779046 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:35.779047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:35.779048 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:35.779132 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.779133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.779752 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.779753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.780763 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:35.780766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.781324 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.781325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:35.781424 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.781428 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:35.781500 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:09:35.781539 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":3,"thread_start_us":128,"threads_started":1}
I20260811 07:09:35.782035 29803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a52d6e3e2741f4b7ef3374c7357fa4" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cbb6676f09414d8e4253a70302dcfe" is_pre_election: true
I20260811 07:09:35.782205 29803 raft_consensus.cc:2454] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 0.
I20260811 07:09:35.782195 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a52d6e3e2741f4b7ef3374c7357fa4" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81158b2eaee74b64adf77759f40c435c" is_pre_election: true
I20260811 07:09:35.782379 29350 raft_consensus.cc:2454] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 0.
I20260811 07:09:35.782490 29467 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe; no voters:
I20260811 07:09:35.782652 29906 raft_consensus.cc:2781] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:09:35.782702 29906 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:09:35.782718 29906 raft_consensus.cc:3037] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.783396 29906 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.783538 29906 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 election: Requested vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501)
I20260811 07:09:35.783679 29803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a52d6e3e2741f4b7ef3374c7357fa4" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cbb6676f09414d8e4253a70302dcfe"
I20260811 07:09:35.783690 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77a52d6e3e2741f4b7ef3374c7357fa4" candidate_uuid: "aeac93dfbf6c4d939482238c243f5bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81158b2eaee74b64adf77759f40c435c"
I20260811 07:09:35.783759 29803 raft_consensus.cc:3037] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.783792 29350 raft_consensus.cc:3037] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:35.784582 29803 raft_consensus.cc:2454] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 1.
I20260811 07:09:35.784751 29467 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe; no voters:
I20260811 07:09:35.784857 29906 raft_consensus.cc:2781] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:35.785025 29906 raft_consensus.cc:686] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 LEADER]: Becoming Leader. State: Replica: aeac93dfbf6c4d939482238c243f5bc2, State: Running, Role: LEADER
I20260811 07:09:35.785156 29906 consensus_queue.cc:237] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:35.785312 29350 raft_consensus.cc:2454] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeac93dfbf6c4d939482238c243f5bc2 in term 1.
I20260811 07:09:35.786087 29088 catalog_manager.cc:5495] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "aeac93dfbf6c4d939482238c243f5bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } health_report { overall_health: UNKNOWN } } }
I20260811 07:09:35.796559 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.794142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45524 (local address 127.3.236.130:38805)
0811 07:09:35.794187 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:35.794189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:35.794199 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:09:35.794268 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:35.794269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:35.794286 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:35.794380 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:09:35.794484 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.794982 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.795720 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:35.795923 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:35.795995 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:35.796166 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:09:35.796308 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:09:35.796412 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:35.796432 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 07:09:35.796458 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:09:35.798385 29350 raft_consensus.cc:1261] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Refusing update from remote peer aeac93dfbf6c4d939482238c243f5bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:35.798424 29803 raft_consensus.cc:1261] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Refusing update from remote peer aeac93dfbf6c4d939482238c243f5bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:35.798738 29906 consensus_queue.cc:1035] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:35.798884 29909 consensus_queue.cc:1035] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:35.802394 29914 mvcc.cc:205] Tried to move back new op lower bound from 7317226192067350528 to 7317226192016703488. Current Snapshot: MvccSnapshot[applied={T|T < 7317226192067350528}]
I20260811 07:09:35.804536 29088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:35.804638 29088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:35.805361 29088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Sending DeleteTablet for 3 replicas of tablet 77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:35.805727 29550 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45514
I20260811 07:09:35.805733 29779 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:52830
I20260811 07:09:35.805943 29930 tablet_replica.cc:330] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: stopping tablet replica
I20260811 07:09:35.806038 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 29166
I20260811 07:09:35.806048 29321 tablet_service.cc:1514] Processing DeleteTablet for tablet 77a52d6e3e2741f4b7ef3374c7357fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:35 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:60328
I20260811 07:09:35.806082 29930 raft_consensus.cc:2229] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:09:35.806203 29930 raft_consensus.cc:2258] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:35.806766 29930 ts_tablet_manager.cc:1905] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:35.809114 29930 ts_tablet_manager.cc:1918] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:35.809193 29930 log.cc:1196] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:35.809535 29930 ts_tablet_manager.cc:1939] T 77a52d6e3e2741f4b7ef3374c7357fa4 P aeac93dfbf6c4d939482238c243f5bc2: Deleting consensus metadata
I20260811 07:09:35.810046 29058 catalog_manager.cc:4841] TS aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): tablet 77a52d6e3e2741f4b7ef3374c7357fa4 (table test-table [id=21e04de56452414e9c01b3a02829c24d]) successfully deleted
I20260811 07:09:35.816670 29929 tablet_replica.cc:330] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: stopping tablet replica
I20260811 07:09:35.816860 29929 raft_consensus.cc:2229] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:35.817063 29929 raft_consensus.cc:2258] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:35.820736 29929 ts_tablet_manager.cc:1905] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:35.822304 29929 ts_tablet_manager.cc:1918] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:35.822402 29929 log.cc:1196] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:35.822713 29929 ts_tablet_manager.cc:1939] T 77a52d6e3e2741f4b7ef3374c7357fa4 P f6cbb6676f09414d8e4253a70302dcfe: Deleting consensus metadata
I20260811 07:09:35.823192 29056 catalog_manager.cc:4841] TS f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): tablet 77a52d6e3e2741f4b7ef3374c7357fa4 (table test-table [id=21e04de56452414e9c01b3a02829c24d]) successfully deleted
W20260811 07:09:35.824945 29062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:36501 (error 108)
I20260811 07:09:35.825373 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 29426
I20260811 07:09:35.825544 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:35.825385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:36501 (local address 127.0.0.1:60332)
0811 07:09:35.825443 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:35.825489 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:36501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:09:35.828466 29062 catalog_manager.cc:4584] TS 81158b2eaee74b64adf77759f40c435c (127.3.236.129:36501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77a52d6e3e2741f4b7ef3374c7357fa4: Network error: Client connection negotiation failed: client connection to 127.3.236.129:36501: connect: Connection refused (error 111)
I20260811 07:09:35.834427 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 29668
I20260811 07:09:35.846483 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 29020
I20260811 07:09:35.862604 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--webserver_interface=127.3.236.190
--webserver_port=44441
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:35.945153 29932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:35.945401 29932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:35.945449 29932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:35.946666 29932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:35.946743 29932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:35.946767 29932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:35.946790 29932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:35.946818 29932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:35.948334 29932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:43623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=44441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:35.948612 29932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:35.948849 29932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:35.953789 29941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.953855 29940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:35.953809 29944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:35.953971 29932 server_base.cc:1034] running on GCE node
I20260811 07:09:35.954236 29932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:35.954528 29932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:35.955699 29932 hybrid_clock.cc:648] HybridClock initialized: now 1786432175955677 us; error 37 us; skew 500 ppm
Aug 11 07:09:35 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432175, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:35.958351 29932 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:35.958777 29932 webserver.cc:460] Webserver started at http://127.3.236.190:44441/ using document root <none> and password file <none>
I20260811 07:09:35.959028 29932 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:35.959098 29932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:35.960476 29932 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:35.961171 29955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:35.961340 29932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:35.961448 29932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "6f499a2da8b54a70acf5e1714ba964b8"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "79b72ec0eaf355b7f1a5692617c20a99"
server_key_iv: "9bbd7e9bb59d22bd8a40f88c5f7d5506"
server_key_version: "encryptionkey@0"
I20260811 07:09:35.961795 29932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:35.970423 29932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:35.972205 29932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:35.972404 29932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:35.976384 29932 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:43623
I20260811 07:09:35.976519 30027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:43623 every 8 connection(s)
I20260811 07:09:35.976831 29932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:35.977829 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 29932
I20260811 07:09:35.978147 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:36501
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=39359
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:35.978609 30029 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:09:35.979265 30029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap starting.
I20260811 07:09:35.988571 30029 log.cc:824] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:35.990947 30029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:09:35.991273 30029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: Bootstrap complete.
I20260811 07:09:35.993718 30029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.993971 30029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Initialized, Role: FOLLOWER
I20260811 07:09:35.994156 30029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.994236 30029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:35.994289 30029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:35.994341 30029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 2 FOLLOWER]: Advancing to term 3
I20260811 07:09:35.995316 30029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.995451 30029 leader_election.cc:302] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f499a2da8b54a70acf5e1714ba964b8; no voters:
I20260811 07:09:35.995790 30029 leader_election.cc:288] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 07:09:35.995831 30035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 07:09:35.995965 30035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [term 3 LEADER]: Becoming Leader. State: Replica: 6f499a2da8b54a70acf5e1714ba964b8, State: Running, Role: LEADER
I20260811 07:09:35.996073 30035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } }
I20260811 07:09:35.996109 30029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:35.996415 30036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:35.996507 30036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:35.996661 30037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f499a2da8b54a70acf5e1714ba964b8. Latest consensus state: current_term: 3 leader_uuid: "6f499a2da8b54a70acf5e1714ba964b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f499a2da8b54a70acf5e1714ba964b8" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43623 } } }
I20260811 07:09:35.996726 30037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:35.998229 30056 catalog_manager.cc:1259] Loaded cluster ID: f34608ec5d2547e4999f04cbaaf580d0
I20260811 07:09:35.998298 30056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: loading cluster ID for follower catalog manager: success
W20260811 07:09:35.998628 30056 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 6f499a2da8b54a70acf5e1714ba964b8: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260811 07:09:35.998697 30056 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260811 07:09:35.998759 30042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:35.999132 30042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:35.999248 30042 catalog_manager.cc:1259] Loaded cluster ID: f34608ec5d2547e4999f04cbaaf580d0
I20260811 07:09:35.999318 30042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 07:09:35.999480 30042 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f247c8669ad google::LogMessage::Fail() at ??:0
@ 0x7f247c86988a google::LogMessage::SendToLog() at ??:0
@ 0x7f247c8664de google::LogMessage::Flush() at ??:0
@ 0x7f247c867a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f24847870f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f24847908b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f247e7340d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f247e729954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f247f58d6db start_thread at ??:0
@ 0x7f247b36461f clone at ??:0
W20260811 07:09:36.075479 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:36.075714 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:36.075754 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:36.077299 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:36.077389 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:36.077440 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:36.079133 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:36501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=39359
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:36.079454 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:36.079725 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:36.085373 30081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.085423 30090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.085382 30085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:36.086104 30031 server_base.cc:1034] running on GCE node
I20260811 07:09:36.086359 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:36.086634 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:36.087808 30031 hybrid_clock.cc:648] HybridClock initialized: now 1786432176087773 us; error 53 us; skew 500 ppm
Aug 11 07:09:36 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:36.090791 30031 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:36.091372 30031 webserver.cc:460] Webserver started at http://127.3.236.129:39359/ using document root <none> and password file <none>
I20260811 07:09:36.091650 30031 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:36.091727 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:36.093369 30031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:36.094175 30099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.094376 30031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:36.094491 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "81158b2eaee74b64adf77759f40c435c"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "fae99dcc5a6c9aceb8134292627922d3"
server_key_iv: "1c334365fea3d5f071e12a48ebfd3146"
server_key_version: "encryptionkey@0"
I20260811 07:09:36.094887 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:36.114518 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:36.117241 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:36.117424 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:36.117694 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:36.118250 30116 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:09:36.120340 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:09:36.120411 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 07:09:36.120471 30031 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:09:36.121215 30031 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:09:36.121279 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 07:09:36.121341 30116 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Bootstrap starting.
I20260811 07:09:36.129418 30031 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:36501
I20260811 07:09:36.129566 30291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:36501 every 8 connection(s)
I20260811 07:09:36.129864 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:09:36.133285 30116 log.cc:824] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:36.134577 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 30031
I20260811 07:09:36.134922 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:38805
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=41783
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:36.136101 30116 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:09:36.136444 30116 tablet_bootstrap.cc:492] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Bootstrap complete.
I20260811 07:09:36.137809 30116 ts_tablet_manager.cc:1397] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.013s
I20260811 07:09:36.139231 30116 raft_consensus.cc:348] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:36.139562 30116 raft_consensus.cc:729] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81158b2eaee74b64adf77759f40c435c, State: Initialized, Role: FOLLOWER
I20260811 07:09:36.139741 30116 consensus_queue.cc:260] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:36.140051 30116 ts_tablet_manager.cc:1428] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
W20260811 07:09:36.236578 30296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:36.236865 30296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:36.236903 30296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:36.238600 30296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:36.238662 30296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:36.238704 30296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:36.240880 30296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:38805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=41783
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:36.241133 30296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:36.241401 30296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:36.246873 30305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.246902 30309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.246920 30306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:36.247546 30296 server_base.cc:1034] running on GCE node
I20260811 07:09:36.247745 30296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:36.248025 30296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:36.249178 30296 hybrid_clock.cc:648] HybridClock initialized: now 1786432176249162 us; error 33 us; skew 500 ppm
Aug 11 07:09:36 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:36.251673 30296 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:36.252251 30296 webserver.cc:460] Webserver started at http://127.3.236.130:41783/ using document root <none> and password file <none>
I20260811 07:09:36.252524 30296 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:36.252571 30296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:36.253973 30296 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:36.254628 30317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.254824 30296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:36.254930 30296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "aeac93dfbf6c4d939482238c243f5bc2"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "556cf77210fd2092bed90dd7a7c404fc"
server_key_iv: "c7ca512f6068c1a3208d0ff9cc50a6b1"
server_key_version: "encryptionkey@0"
I20260811 07:09:36.255357 30296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:36.279801 30296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:36.284032 30296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:36.284211 30296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:36.284502 30296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:36.284833 30296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:36.284869 30296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.284893 30296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:36.284948 30296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260811 07:09:36.287035 30294 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:09:36.130567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:38225)
0811 07:09:36.130757 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.130794 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:36.131287 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:36.286792 (+155505us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
W20260811 07:09:36.287330 30292 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:43623 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:43623: Client connection negotiation failed: client connection to 127.3.236.190:43623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:09:36.287770 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.287621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:36193)
0811 07:09:36.287687 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.287737 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:09:36.288151 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.288039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:55529)
0811 07:09:36.288093 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.288123 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 07:09:36.288316 30292 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:09:36.291641 30296 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:38805
I20260811 07:09:36.291723 30497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:38805 every 8 connection(s)
I20260811 07:09:36.292107 30296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:09:36.292521 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 30296
I20260811 07:09:36.292797 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:35117
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:43623
--webserver_port=45231
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:09:36.293283 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.292874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:43621)
0811 07:09:36.293061 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.293137 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
W20260811 07:09:36.294981 30498 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:43623 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:43623: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
I20260811 07:09:36.295308 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.295208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:60727)
0811 07:09:36.295259 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.295289 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:09:36.295679 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.295523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:51665)
0811 07:09:36.295612 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.295646 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 07:09:36.295850 30498 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:09:36.377383 30501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:36.377599 30501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:36.377622 30501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:36.378850 30501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:36.378893 30501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:36.378921 30501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:36.380424 30501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:35117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=45231
--tserver_master_addrs=127.3.236.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:36.380681 30501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:36.380928 30501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:36.386188 30514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.386216 30510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:36.386229 30511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:36.386480 30501 server_base.cc:1034] running on GCE node
I20260811 07:09:36.386744 30501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:36.387014 30501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:36.388196 30501 hybrid_clock.cc:648] HybridClock initialized: now 1786432176388183 us; error 44 us; skew 500 ppm
Aug 11 07:09:36 dist-test-slave-2d19 krb5kdc[28100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:36.391099 30501 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:36.391501 30501 webserver.cc:460] Webserver started at http://127.3.236.131:45231/ using document root <none> and password file <none>
I20260811 07:09:36.391677 30501 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:36.391717 30501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:36.393119 30501 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:36.393679 30525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.393834 30501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:09:36.393947 30501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "f6cbb6676f09414d8e4253a70302dcfe"
format_stamp: "Formatted at 2026-08-11 07:09:34 on dist-test-slave-2d19"
server_key: "986649d2973d16e9e857441a3111bb67"
server_key_iv: "22355f40de96d5e143b2497b8c2a1d51"
server_key_version: "encryptionkey@0"
I20260811 07:09:36.394292 30501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:36.407629 30501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:36.409611 30501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:36.409724 30501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:36.409901 30501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:36.410226 30501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:36.410262 30501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.410283 30501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:36.410334 30501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:36.416206 30501 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35117
I20260811 07:09:36.416316 30704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35117 every 8 connection(s)
I20260811 07:09:36.416656 30501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:09:36.417711 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.417339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:42243)
0811 07:09:36.417501 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.417571 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
W20260811 07:09:36.418138 30705 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:43623 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:43623: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
I20260811 07:09:36.418459 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.418367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:32877)
0811 07:09:36.418411 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.418439 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 07:09:36.418759 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:36.418683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:43411)
0811 07:09:36.418722 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:36.418744 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 07:09:36.418876 30705 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:09:36.419030 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 30501
I20260811 07:09:37.288966 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.288706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:49567)
0811 07:09:37.288844 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:37.288884 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260811 07:09:37.296396 30710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.296185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:39981)
0811 07:09:37.296310 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:37.296348 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260811 07:09:37.419397 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.419183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:33687)
0811 07:09:37.419304 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:37.419341 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260811 07:09:37.993450 30716 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:37.993651 30716 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:37.993992 30716 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
Aug 11 07:09:37 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.131@KRBTEST.COM
Aug 11 07:09:37 dist-test-slave-2d19 krb5kdc[28100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.130@KRBTEST.COM
I20260811 07:09:38.005446 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.994251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35117 (local address 127.3.236.129:59759)
0811 07:09:37.994526 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:37.994535 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:37.994584 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:37.995473 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:37.995474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:37.995838 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:37.996028 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:37.996033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.997265 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.997266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:37.998177 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:37.998180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.998958 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:37.999031 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:37.999693 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:37.999698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:38.001428 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:38.003552 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:38.003559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:38.003561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:38.003858 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:38.004154 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:38.004155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:38.004349 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:38.004841 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:38.004848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:38.005020 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 07:09:38.005141 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 07:09:38.005320 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":38,"thread_start_us":63,"threads_started":1}
I20260811 07:09:38.005594 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.994408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:59759 (local address 127.3.236.131:35117)
0811 07:09:37.994594 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:37.994608 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:37.994634 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:09:37.995195 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:37.995197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:37.995245 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:37.995365 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:37.996162 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.997170 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998283 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.998865 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998990 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:38.001586 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:38.001590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:38.001591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:38.001617 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:38.003458 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:38.003969 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:38.003972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:38.003972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:38.004075 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004430 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:38.004431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:38.004432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:38.004763 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:38.004885 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:38.005315 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 07:09:38.005467 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260811 07:09:38.005738 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.994365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:42897 (local address 127.3.236.130:38805)
0811 07:09:37.994608 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:37.994621 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:37.994764 (+ 143us) server_negotiation.cc:407] Connection header received
0811 07:09:37.995214 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:37.995216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:37.995255 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:37.995411 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:37.996149 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.997100 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998267 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.998753 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998910 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:38.002037 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:38.002041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:38.002041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:38.002054 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:38.004099 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004592 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:38.004595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:38.004595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:38.004677 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004891 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:38.004892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:38.004892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:38.005195 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:38.005319 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:38.005492 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 07:09:38.005641 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260811 07:09:38.005612 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:37.994209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:38805 (local address 127.3.236.129:42897)
0811 07:09:37.994394 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:37.994408 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:37.994472 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:37.995458 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:37.995460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:37.995840 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:37.996028 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:37.996033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.997230 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.997230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:37.998178 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:37.998181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:37.998887 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:37.998889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:37.999028 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:37.999645 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:37.999658 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:38.001904 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:38.004215 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:38.004217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:38.004488 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:38.004748 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:38.004749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:38.004749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:38.004825 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:38.005295 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:38.005296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:38.005342 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 07:09:38.005421 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:09:38.005516 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":53,"thread_start_us":88,"threads_started":1}
W20260811 07:09:38.007014 30104 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
W20260811 07:09:38.007098 30102 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:38.007157 30102 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81158b2eaee74b64adf77759f40c435c; no voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe
I20260811 07:09:38.007295 30716 raft_consensus.cc:2726] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:09:38.289808 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:38.289591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:49029)
0811 07:09:38.289677 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:38.289727 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 07:09:38.297104 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:38.296784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:38555)
0811 07:09:38.296915 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:38.297059 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260811 07:09:38.420077 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:38.419821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:59585)
0811 07:09:38.419971 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:38.420027 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260811 07:09:39.290592 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:39.290325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:48451)
0811 07:09:39.290502 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:39.290542 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
I20260811 07:09:39.297864 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:39.297588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:50121)
0811 07:09:39.297751 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:39.297788 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260811 07:09:39.420789 30728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:39.420569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:58545)
0811 07:09:39.420702 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:39.420744 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260811 07:09:40.087880 30729 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:40.088014 30729 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:40.088294 30729 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
W20260811 07:09:40.088840 30102 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
W20260811 07:09:40.088928 30104 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:40.089049 30104 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81158b2eaee74b64adf77759f40c435c; no voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe
I20260811 07:09:40.089159 30729 raft_consensus.cc:2726] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:09:40.291404 30731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:40.291155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:40391)
0811 07:09:40.291317 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:40.291353 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260811 07:09:40.298534 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:40.298307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:39157)
0811 07:09:40.298426 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:40.298462 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260811 07:09:40.421566 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:40.421297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:40733)
0811 07:09:40.421462 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:40.421506 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260811 07:09:41.292217 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:41.291962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:43065)
0811 07:09:41.292126 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:41.292170 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260811 07:09:41.299155 30737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:41.298967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:33691)
0811 07:09:41.299078 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:41.299113 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260811 07:09:41.422384 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:41.422151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:47743)
0811 07:09:41.422292 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:41.422335 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260811 07:09:42.292971 30741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:42.292729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:49021)
0811 07:09:42.292887 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:42.292929 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260811 07:09:42.299813 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:42.299583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:58291)
0811 07:09:42.299729 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:42.299768 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260811 07:09:42.423079 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:42.422859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:51827)
0811 07:09:42.422987 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:42.423029 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260811 07:09:42.444145 30748 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:42.444311 30748 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:42.444546 30748 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
W20260811 07:09:42.444950 30102 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
W20260811 07:09:42.445035 30104 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:42.445120 30104 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81158b2eaee74b64adf77759f40c435c; no voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe
I20260811 07:09:42.445218 30748 raft_consensus.cc:2726] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:09:43.293656 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:43.293441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:38089)
0811 07:09:43.293574 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:43.293613 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260811 07:09:43.300560 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:43.300353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:50765)
0811 07:09:43.300482 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:43.300516 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260811 07:09:43.423801 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:43.423570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:53949)
0811 07:09:43.423719 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:43.423755 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260811 07:09:44.294402 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:44.294169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:34895)
0811 07:09:44.294324 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:44.294362 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260811 07:09:44.301214 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:44.300998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:34217)
0811 07:09:44.301138 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:44.301173 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260811 07:09:44.424624 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:44.424381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:56793)
0811 07:09:44.424529 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:44.424574 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260811 07:09:45.295133 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:45.294900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:50297)
0811 07:09:45.295050 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:45.295089 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260811 07:09:45.302098 30763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:45.301808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:56023)
0811 07:09:45.302013 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:45.302046 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":103,"threads_started":1}
I20260811 07:09:45.425365 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:45.425123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:45053)
0811 07:09:45.425277 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:45.425317 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260811 07:09:46.295908 30766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:46.295662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:55213)
0811 07:09:46.295820 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:46.295863 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260811 07:09:46.302740 30769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:46.302509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:33393)
0811 07:09:46.302644 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:46.302692 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 07:09:46.323035 30774 raft_consensus.cc:482] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:46.323150 30774 raft_consensus.cc:504] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cbb6676f09414d8e4253a70302dcfe" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35117 } } peers { permanent_uuid: "aeac93dfbf6c4d939482238c243f5bc2" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 38805 } } peers { permanent_uuid: "81158b2eaee74b64adf77759f40c435c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 36501 } }
I20260811 07:09:46.323323 30774 leader_election.cc:288] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117), aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805)
W20260811 07:09:46.323786 30102 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6cbb6676f09414d8e4253a70302dcfe (127.3.236.131:35117): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
W20260811 07:09:46.323891 30104 leader_election.cc:341] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aeac93dfbf6c4d939482238c243f5bc2 (127.3.236.130:38805): Not found: Tablet not found: 77a52d6e3e2741f4b7ef3374c7357fa4
I20260811 07:09:46.323978 30104 leader_election.cc:302] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81158b2eaee74b64adf77759f40c435c; no voters: aeac93dfbf6c4d939482238c243f5bc2, f6cbb6676f09414d8e4253a70302dcfe
I20260811 07:09:46.324064 30774 raft_consensus.cc:2726] T 77a52d6e3e2741f4b7ef3374c7357fa4 P 81158b2eaee74b64adf77759f40c435c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:09:46.426177 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:46.425913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:35975)
0811 07:09:46.426078 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:46.426124 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260811 07:09:47.296625 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:47.296408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:43071)
0811 07:09:47.296540 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:47.296580 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260811 07:09:47.303352 30779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:47.303151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:37545)
0811 07:09:47.303275 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:47.303310 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260811 07:09:47.426895 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:47.426661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:55019)
0811 07:09:47.426800 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:47.426845 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260811 07:09:48.297288 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:48.297071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:55281)
0811 07:09:48.297201 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:48.297241 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260811 07:09:48.304037 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:48.303792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:48033)
0811 07:09:48.303938 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:48.303970 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260811 07:09:48.427785 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:48.427537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:43735)
0811 07:09:48.427692 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:48.427736 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260811 07:09:49.298074 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:49.297806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:54537)
0811 07:09:49.297985 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:49.298031 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":143,"threads_started":1}
I20260811 07:09:49.304742 30789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:49.304504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:55793)
0811 07:09:49.304661 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:49.304698 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260811 07:09:49.428519 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:49.428300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:53965)
0811 07:09:49.428434 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:49.428469 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260811 07:09:50.298697 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:50.298467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:51627)
0811 07:09:50.298603 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:50.298651 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260811 07:09:50.305413 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:50.305210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:35743)
0811 07:09:50.305331 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:50.305370 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260811 07:09:50.429221 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:50.429002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:54381)
0811 07:09:50.429133 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:50.429170 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260811 07:09:51.299460 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:51.299246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.129:37559)
0811 07:09:51.299376 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:51.299416 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260811 07:09:51.306075 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:51.305881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.130:58621)
0811 07:09:51.306007 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:51.306036 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260811 07:09:51.427392 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 30031
I20260811 07:09:51.439656 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:51.439421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43623 (local address 127.3.236.131:43281)
0811 07:09:51.439560 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:51.439606 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:43623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260811 07:09:51.444063 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 30296
I20260811 07:09:51.459478 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 30501
2026-08-11T07:09:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18569 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:09:51 dist-test-slave-2d19 krb5kdc[30807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:09:51 dist-test-slave-2d19 krb5kdc[30807](info): set up 2 sockets
Aug 11 07:09:51 dist-test-slave-2d19 krb5kdc[30807](info): commencing operation
krb5kdc: starting...
W20260811 07:09:52.760913 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:09:52 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:09:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:09:52.886601 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:52.969576 30827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:52.969784 30827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:52.969805 30827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:52.970955 30827 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:09:52.970996 30827 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:09:52.971009 30827 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:09:52.971021 30827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:09:52.971035 30827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:52.971047 30827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:52.971099 30827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:09:52.971119 30827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:09:52.972615 30827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:52.972862 30827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:52.973115 30827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:52.978370 30836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:52.978338 30841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:52.978353 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:52.978408 30827 server_base.cc:1034] running on GCE node
I20260811 07:09:52.978842 30827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:52.979102 30827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:52.980247 30827 hybrid_clock.cc:648] HybridClock initialized: now 1786432192980228 us; error 34 us; skew 500 ppm
Aug 11 07:09:52 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:52.982781 30827 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:09:52.983274 30827 webserver.cc:460] Webserver started at http://127.3.236.190:46213/ using document root <none> and password file <none>
I20260811 07:09:52.983501 30827 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:52.983548 30827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:52.983714 30827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:52.984851 30827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "d42db9fec6a344b09de9784bfcadd28c"
format_stamp: "Formatted at 2026-08-11 07:09:52 on dist-test-slave-2d19"
server_key: "601e7197b78bba474db727404c59e398"
server_key_iv: "7bf5117ea760a6bda0722ec6e66e576b"
server_key_version: "encryptionkey@0"
I20260811 07:09:52.985239 30827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "d42db9fec6a344b09de9784bfcadd28c"
format_stamp: "Formatted at 2026-08-11 07:09:52 on dist-test-slave-2d19"
server_key: "601e7197b78bba474db727404c59e398"
server_key_iv: "7bf5117ea760a6bda0722ec6e66e576b"
server_key_version: "encryptionkey@0"
I20260811 07:09:52.986708 30827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:09:52.987489 30850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:52.987682 30827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:52.987794 30827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "d42db9fec6a344b09de9784bfcadd28c"
format_stamp: "Formatted at 2026-08-11 07:09:52 on dist-test-slave-2d19"
server_key: "601e7197b78bba474db727404c59e398"
server_key_iv: "7bf5117ea760a6bda0722ec6e66e576b"
server_key_version: "encryptionkey@0"
I20260811 07:09:52.987874 30827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:52.995455 30827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:52.997318 30827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:52.997478 30827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:53.001735 30827 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35667
I20260811 07:09:53.001787 30928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35667 every 8 connection(s)
I20260811 07:09:53.002151 30827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:09:53.003024 30931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:53.005401 30931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap starting.
I20260811 07:09:53.006003 30931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:53.006318 30931 log.cc:824] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:53.006985 30931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: No bootstrap required, opened a new log
I20260811 07:09:53.008667 30931 raft_consensus.cc:348] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:09:53.008803 30931 raft_consensus.cc:374] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:53.008826 30931 raft_consensus.cc:729] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Initialized, Role: FOLLOWER
I20260811 07:09:53.008996 30931 consensus_queue.cc:260] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:09:53.009058 30931 raft_consensus.cc:388] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:09:53.009104 30931 raft_consensus.cc:482] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:09:53.009155 30931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:53.009692 30931 raft_consensus.cc:504] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:09:53.009806 30931 leader_election.cc:302] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d42db9fec6a344b09de9784bfcadd28c; no voters:
I20260811 07:09:53.009989 30931 leader_election.cc:288] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:09:53.010054 30934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:53.010238 30934 raft_consensus.cc:686] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 LEADER]: Becoming Leader. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Running, Role: LEADER
I20260811 07:09:53.010275 30931 sys_catalog.cc:564] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:09:53.010344 30934 consensus_queue.cc:237] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:09:53.010754 30937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:09:53.010768 30939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d42db9fec6a344b09de9784bfcadd28c. Latest consensus state: current_term: 1 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:09:53.010951 30939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:53.011186 30937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:09:53.011333 30947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:09:53.011838 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 30827
I20260811 07:09:53.011920 30947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:09:53.011952 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:09:53.012185 4018 external_mini_cluster.cc:1428] Setting key 4a345bbd9da1906d679d0d6a6673c9b2
I20260811 07:09:53.013692 30947 catalog_manager.cc:1347] Generated new cluster ID: 96e521775eac4ac288958f1f85032812
I20260811 07:09:53.013747 30947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:53.024503 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.012741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.3.236.190:35667)
0811 07:09:53.012945 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.012955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.013206 (+ 251us) server_negotiation.cc:407] Connection header received
0811 07:09:53.013356 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.013364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.013416 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.013566 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:53.014275 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.015270 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.016054 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.016598 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.016716 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.019156 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:53.019161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:53.019162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:53.019175 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:53.022081 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.022831 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.022835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.022835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.022952 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.023171 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.023172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.023173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.023558 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:53.023695 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.023923 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.024182 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260811 07:09:53.030346 30947 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:09:53.031140 30947 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:09:53.049266 30947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Generated new TSK 0
I20260811 07:09:53.049497 30947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:09:53.088095 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:53.176440 30972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:53.176661 30972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:53.176702 30972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:53.177860 30972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:53.177927 30972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:53.178010 30972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:09:53.179519 30972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:53.179812 30972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:53.180053 30972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:53.185235 30981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.185328 30980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.185321 30984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:53.185551 30972 server_base.cc:1034] running on GCE node
I20260811 07:09:53.185736 30972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:53.185943 30972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:53.187124 30972 hybrid_clock.cc:648] HybridClock initialized: now 1786432193187095 us; error 52 us; skew 500 ppm
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:53.189822 30972 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:09:53.190389 30972 webserver.cc:460] Webserver started at http://127.3.236.129:38717/ using document root <none> and password file <none>
I20260811 07:09:53.190691 30972 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:53.190760 30972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:53.190897 30972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:53.192088 30972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "8e608cb5a7d143c08591c538250f02ce"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "0139a370a7d840afea486c163ab17bd1"
server_key_iv: "cf798ec1dd022664c2efaaea4413f676"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.192520 30972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "8e608cb5a7d143c08591c538250f02ce"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "0139a370a7d840afea486c163ab17bd1"
server_key_iv: "cf798ec1dd022664c2efaaea4413f676"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.193853 30972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:53.194638 30994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.194797 30972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:09:53.194891 30972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "8e608cb5a7d143c08591c538250f02ce"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "0139a370a7d840afea486c163ab17bd1"
server_key_iv: "cf798ec1dd022664c2efaaea4413f676"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.194947 30972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:53.205017 30972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:53.206772 30972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:53.206881 30972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:53.207073 30972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:53.207327 30972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:53.207358 30972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.207379 30972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:53.207392 30972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.213766 30972 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:45957
I20260811 07:09:53.213821 31182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:45957 every 8 connection(s)
I20260811 07:09:53.214135 30972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:09:53.214340 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 30972
I20260811 07:09:53.214452 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:09:53.214721 4018 external_mini_cluster.cc:1428] Setting key 2b13895a8df26a85c062463c109b51fb
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
I20260811 07:09:53.224718 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.215221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:43423 (local address 127.3.236.190:35667)
0811 07:09:53.215262 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.215264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.215274 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:09:53.215714 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.215716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.215732 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.215806 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:53.216806 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.217548 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.219067 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.219266 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.219323 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.221767 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:53.221768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:53.221769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:53.221776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:53.223251 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.223686 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.223687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.223688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.223748 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.224017 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.224018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.224018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.224129 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:53.224202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.224539 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.224627 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:09:53.224901 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.214910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:43423)
0811 07:09:53.215091 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.215132 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.215613 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.215860 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.215863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.216520 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:53.216712 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.216718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.217647 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.217648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.218966 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.218970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.219357 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.219358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.219507 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.220024 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:53.220036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:53.221666 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:53.223356 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.223364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.223366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.223605 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.223821 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.223822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.223822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.223933 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.224205 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:53.224213 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:53.224387 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.224513 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.224669 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260811 07:09:53.225437 31185 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
I20260811 07:09:53.225565 31185 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:53.225751 31185 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:09:53.226269 30873 ts_manager.cc:194] Registered new tserver with Master: 8e608cb5a7d143c08591c538250f02ce (127.3.236.129:45957)
I20260811 07:09:53.226989 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:43423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:09:53.243314 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:53.326889 31192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:53.327147 31192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:53.327171 31192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:53.328240 31192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:53.328302 31192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:53.328328 31192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:09:53.329725 31192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:53.329985 31192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:53.330166 31192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:53.335326 31201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.335256 31200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.335255 31206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:53.335574 31192 server_base.cc:1034] running on GCE node
I20260811 07:09:53.335763 31192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:53.336010 31192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:53.337180 31192 hybrid_clock.cc:648] HybridClock initialized: now 1786432193337153 us; error 39 us; skew 500 ppm
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:53.339975 31192 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:09:53.340448 31192 webserver.cc:460] Webserver started at http://127.3.236.130:40549/ using document root <none> and password file <none>
I20260811 07:09:53.340651 31192 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:53.340693 31192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:53.340782 31192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:53.341782 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "14720846d6472e0b9ef38f99122ca7f1"
server_key_iv: "0ce306fb2750de8f3e3aa472d8dc278f"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.342097 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "14720846d6472e0b9ef38f99122ca7f1"
server_key_iv: "0ce306fb2750de8f3e3aa472d8dc278f"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.343389 31192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:09:53.344157 31224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.344357 31192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:09:53.344441 31192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "14720846d6472e0b9ef38f99122ca7f1"
server_key_iv: "0ce306fb2750de8f3e3aa472d8dc278f"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.344492 31192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:53.378525 31192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:53.380663 31192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:53.380784 31192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:53.380992 31192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:53.381343 31192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:53.381381 31192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.381403 31192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:53.381415 31192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.387395 31192 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:44809
I20260811 07:09:53.387503 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:44809 every 8 connection(s)
I20260811 07:09:53.387816 31192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:09:53.389055 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 31192
I20260811 07:09:53.389184 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:09:53.389384 4018 external_mini_cluster.cc:1428] Setting key 3e58226cfc6d0421b4d9a5b338068ddb
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:53.398694 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.388648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:57987 (local address 127.3.236.190:35667)
0811 07:09:53.388739 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.388741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.388932 (+ 191us) server_negotiation.cc:407] Connection header received
0811 07:09:53.389669 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.389670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.389685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.389757 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:53.390935 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.391715 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.392951 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.393156 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.393233 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.395673 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:53.395675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:53.395676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:53.395684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:53.397192 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.397647 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.397648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.397648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.397713 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.398012 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.398014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.398014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.398142 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:53.398218 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.398514 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.398603 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:09:53.398891 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.388635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:57987)
0811 07:09:53.388803 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.388848 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.389514 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.389847 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.389852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.390552 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:53.390846 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.390853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.391819 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.391820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.392835 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.392838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.393256 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.393257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.393401 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.393937 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:53.393947 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:53.395551 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:53.397300 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.397308 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.397311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.397542 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.397792 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.397793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.397793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.397941 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.398221 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:53.398229 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:53.398370 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.398476 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.398649 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260811 07:09:53.399405 31400 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
I20260811 07:09:53.399529 31400 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:53.399746 31400 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:09:53.400187 30873 ts_manager.cc:194] Registered new tserver with Master: 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809)
I20260811 07:09:53.400586 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:57987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:09:53.418867 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:09:53.503904 31408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:09:53.504160 31408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:09:53.504205 31408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:09:53.505402 31408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:09:53.505472 31408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:09:53.505518 31408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:09:53.507007 31408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:09:53.507273 31408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:09:53.507505 31408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:09:53.513124 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.513080 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:09:53.513103 31424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:09:53.513396 31408 server_base.cc:1034] running on GCE node
I20260811 07:09:53.513602 31408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:09:53.513870 31408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:09:53.515057 31408 hybrid_clock.cc:648] HybridClock initialized: now 1786432193515061 us; error 51 us; skew 500 ppm
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:09:53.517679 31408 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:09:53.518117 31408 webserver.cc:460] Webserver started at http://127.3.236.131:37399/ using document root <none> and password file <none>
I20260811 07:09:53.518349 31408 fs_manager.cc:359] Metadata directory not provided
I20260811 07:09:53.518419 31408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:09:53.518581 31408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:09:53.519644 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "4cf64a23ed0fd1d127191b24250fa220"
server_key_iv: "544e18d1063f64f77e6fa58f07a2629e"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.520013 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "4cf64a23ed0fd1d127191b24250fa220"
server_key_iv: "544e18d1063f64f77e6fa58f07a2629e"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.521449 31408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:09:53.522257 31433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.522472 31408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:09:53.522586 31408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "4cf64a23ed0fd1d127191b24250fa220"
server_key_iv: "544e18d1063f64f77e6fa58f07a2629e"
server_key_version: "encryptionkey@0"
I20260811 07:09:53.522672 31408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:09:53.546072 31408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:09:53.548240 31408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:09:53.548472 31408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:09:53.548759 31408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:09:53.549149 31408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:09:53.549218 31408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.549280 31408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:09:53.549306 31408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:09:53.556104 31408 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:38059
I20260811 07:09:53.556156 31613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:38059 every 8 connection(s)
I20260811 07:09:53.556641 31408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:09:53 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432193, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:09:53.564600 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 31408
I20260811 07:09:53.564713 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:09:53.564888 4018 external_mini_cluster.cc:1428] Setting key 66dc6009c725fbfb0d33310e0f25880a
I20260811 07:09:53.566857 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.557408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:39013 (local address 127.3.236.190:35667)
0811 07:09:53.557457 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.557459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.557698 (+ 239us) server_negotiation.cc:407] Connection header received
0811 07:09:53.558283 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.558284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.558298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.558348 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:53.559329 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.559836 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.560903 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.561111 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.561200 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.563477 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:53.563479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:53.563480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:53.563488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:53.565324 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.565835 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.565836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.565837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.565908 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.566188 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.566189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.566189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.566308 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:53.566382 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.566685 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.566768 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:09:53.567068 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.557395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:39013)
0811 07:09:53.557573 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.557611 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.558162 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.558420 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.558425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.559018 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:09:53.559215 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.559220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.559945 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.559946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.560809 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.560812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.561214 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.561215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.561335 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.561865 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:09:53.561874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:09:53.563334 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:09:53.565430 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.565437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.565440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.565760 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.565990 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:09:53.565991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:09:53.565991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:09:53.566110 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:09:53.566386 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:09:53.566401 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:09:53.566541 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.566646 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.566810 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260811 07:09:53.567488 31614 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
I20260811 07:09:53.567580 31614 heartbeater.cc:464] Registering TS with master...
I20260811 07:09:53.567728 31614 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:09:53.568084 30873 ts_manager.cc:194] Registered new tserver with Master: bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:09:53.568480 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:39013
I20260811 07:09:53.576810 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:09:53.584367 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.578208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.3.236.190:35667)
0811 07:09:53.578277 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.578279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.578393 (+ 114us) server_negotiation.cc:407] Connection header received
0811 07:09:53.578437 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.578438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.578447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.578500 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:09:53.579059 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.579526 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.580442 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.580624 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.580674 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.581887 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:09:53.581888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:09:53.581888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:09:53.581894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:09:53.583305 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.583612 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.583613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.583613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.583663 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:09:53.583837 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:09:53.583837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:09:53.583838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:09:53.583970 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:09:53.584060 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.584170 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.584228 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:09:53.586126 30873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:09:53.588357 30873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:09:53.595781 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:45957 (local address 127.0.0.1:45134)
0811 07:09:53.591820 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.591830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.591870 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.592211 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.592214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.592217 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:53.592309 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.592311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593668 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.593669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.594782 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.594784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.595517 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.595518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.595596 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.595600 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.595650 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.595692 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":31,"thread_start_us":91,"threads_started":1}
I20260811 07:09:53.595804 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.3.236.129:45957)
0811 07:09:53.592043 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.592047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.592056 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:53.592090 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.592092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.592104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.592189 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:53.592411 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593551 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.594870 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.595431 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.595501 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.595568 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.595677 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.595726 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260811 07:09:53.595997 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44809 (local address 127.0.0.1:40774)
0811 07:09:53.591560 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.591602 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.591665 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.592337 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.592338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.592341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:53.592398 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.592400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593336 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.593337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.594438 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.594440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.595383 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.595384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.595501 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.595517 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.595588 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.595671 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":74,"threads_started":1}
I20260811 07:09:53.596493 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40774 (local address 127.3.236.130:44809)
0811 07:09:53.592048 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.592052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.592062 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:09:53.592096 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.592098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.592111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.592219 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:53.592493 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593213 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.594537 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.595284 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.595360 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.595413 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.595743 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.595787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":93,"threads_started":1}
I20260811 07:09:53.596869 31073 tablet_service.cc:1467] Processing CreateTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb (DEFAULT_TABLE table=test-table [id=488550653e034f419f92d89b60b1fcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:53.597019 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:38059 (local address 127.0.0.1:55470)
0811 07:09:53.591800 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.591809 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.591855 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.592624 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.592625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.592627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:53.592662 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.592663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593855 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.593856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.595799 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.595801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.596774 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.596775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.596832 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.596835 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.596907 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.596945 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":108,"threads_started":1}
I20260811 07:09:53.597100 31298 tablet_service.cc:1467] Processing CreateTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb (DEFAULT_TABLE table=test-table [id=488550653e034f419f92d89b60b1fcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:53.597268 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.591717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.3.236.131:38059)
0811 07:09:53.592391 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.592396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.592405 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:53.592453 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.592456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.592472 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.592547 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:53.592740 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.593749 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.595898 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.596615 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.596687 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.596739 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.596928 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.596971 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":91,"threads_started":1}
I20260811 07:09:53.597293 31073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cca963f51ba4900bc4e81e04a76e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:53.597448 31298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cca963f51ba4900bc4e81e04a76e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:53.597849 31510 tablet_service.cc:1467] Processing CreateTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb (DEFAULT_TABLE table=test-table [id=488550653e034f419f92d89b60b1fcc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:09:53.598100 31510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cca963f51ba4900bc4e81e04a76e9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:09:53.600761 31643 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Bootstrap starting.
I20260811 07:09:53.601158 31644 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Bootstrap starting.
I20260811 07:09:53.601649 31643 tablet_bootstrap.cc:654] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:53.601847 31644 tablet_bootstrap.cc:654] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:53.602286 31643 log.cc:824] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:53.602474 31644 log.cc:824] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:53.602888 31646 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Bootstrap starting.
I20260811 07:09:53.603192 31643 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: No bootstrap required, opened a new log
I20260811 07:09:53.603281 31643 ts_tablet_manager.cc:1397] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:09:53.603541 31646 tablet_bootstrap.cc:654] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Neither blocks nor log segments found. Creating new log.
I20260811 07:09:53.604015 31646 log.cc:824] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Log is configured to *not* fsync() on all Append() calls
I20260811 07:09:53.604597 31644 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: No bootstrap required, opened a new log
I20260811 07:09:53.604655 31644 ts_tablet_manager.cc:1397] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:09:53.604892 31646 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: No bootstrap required, opened a new log
I20260811 07:09:53.604998 31646 ts_tablet_manager.cc:1397] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:09:53.605413 31643 raft_consensus.cc:348] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.605607 31643 raft_consensus.cc:374] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:53.605656 31643 raft_consensus.cc:729] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3c3dd6268246a2bc14b4078f4ad52c, State: Initialized, Role: FOLLOWER
I20260811 07:09:53.605791 31643 consensus_queue.cc:260] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.606108 31643 ts_tablet_manager.cc:1428] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:09:53.606132 31614 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
I20260811 07:09:53.606462 31644 raft_consensus.cc:348] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.606653 31644 raft_consensus.cc:374] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:53.606693 31644 raft_consensus.cc:729] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, State: Initialized, Role: FOLLOWER
I20260811 07:09:53.606819 31644 consensus_queue.cc:260] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.607124 31644 ts_tablet_manager.cc:1428] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 07:09:53.607270 31400 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
I20260811 07:09:53.607538 31646 raft_consensus.cc:348] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.607735 31646 raft_consensus.cc:374] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:09:53.607779 31646 raft_consensus.cc:729] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Initialized, Role: FOLLOWER
I20260811 07:09:53.607889 31646 consensus_queue.cc:260] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.608188 31646 ts_tablet_manager.cc:1428] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:09:53.608197 31185 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
W20260811 07:09:53.638450 31403 tablet.cc:2369] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:09:53.715054 31186 tablet.cc:2369] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:09:53.783667 31655 raft_consensus.cc:482] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:09:53.783855 31655 raft_consensus.cc:504] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.784359 31655 leader_election.cc:288] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:09:53.787626 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.784633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:38075 (local address 127.3.236.130:44809)
0811 07:09:53.784689 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.784691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.784795 (+ 104us) server_negotiation.cc:407] Connection header received
0811 07:09:53.784842 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.784843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.784856 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.784927 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:53.785136 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.785701 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.786751 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.787208 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.787315 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.787324 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.787494 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.787542 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 07:09:53.787642 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.784660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:60953 (local address 127.3.236.131:38059)
0811 07:09:53.784723 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.784725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.784880 (+ 155us) server_negotiation.cc:407] Connection header received
0811 07:09:53.784946 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.784948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.784957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.785018 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:09:53.785227 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.785686 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.786837 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.787210 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.787304 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.787315 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.787522 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.787562 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:09:53.787653 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.784543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44809 (local address 127.3.236.129:38075)
0811 07:09:53.784704 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.784717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.784794 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.784942 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.784943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.784945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:53.785041 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.785043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.785836 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.785837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.786649 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.786651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.787337 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.787338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.787412 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.787415 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.787461 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.787527 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":18,"thread_start_us":78,"threads_started":1}
I20260811 07:09:53.787669 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.784543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:38059 (local address 127.3.236.129:60953)
0811 07:09:53.784754 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.784784 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 07:09:53.784884 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:09:53.785040 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:09:53.785041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:09:53.785042 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:09:53.785144 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.785145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.785814 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.785815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.786735 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:09:53.786739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.787301 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.787303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:09:53.787414 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.787418 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:09:53.787501 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:09:53.787547 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260811 07:09:53.788141 31331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cca963f51ba4900bc4e81e04a76e9eb" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" is_pre_election: true
I20260811 07:09:53.788152 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cca963f51ba4900bc4e81e04a76e9eb" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" is_pre_election: true
I20260811 07:09:53.788328 31331 raft_consensus.cc:2454] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 0.
I20260811 07:09:53.788328 31547 raft_consensus.cc:2454] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 0.
I20260811 07:09:53.788560 31001 leader_election.cc:302] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce, bd3c3dd6268246a2bc14b4078f4ad52c; no voters:
I20260811 07:09:53.788712 31655 raft_consensus.cc:2781] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:09:53.788771 31655 raft_consensus.cc:482] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:09:53.788801 31655 raft_consensus.cc:3037] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:53.789523 31655 raft_consensus.cc:504] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.789674 31655 leader_election.cc:288] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 election: Requested vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:09:53.789852 31547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cca963f51ba4900bc4e81e04a76e9eb" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
I20260811 07:09:53.789917 31331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cca963f51ba4900bc4e81e04a76e9eb" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
I20260811 07:09:53.789968 31547 raft_consensus.cc:3037] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:53.790004 31331 raft_consensus.cc:3037] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:09:53.790843 31331 raft_consensus.cc:2454] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 1.
I20260811 07:09:53.790910 31547 raft_consensus.cc:2454] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 1.
I20260811 07:09:53.791054 31001 leader_election.cc:302] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce, bd3c3dd6268246a2bc14b4078f4ad52c; no voters:
I20260811 07:09:53.791200 31655 raft_consensus.cc:2781] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:09:53.791425 31655 raft_consensus.cc:686] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 LEADER]: Becoming Leader. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Running, Role: LEADER
I20260811 07:09:53.791509 31655 consensus_queue.cc:237] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:09:53.792425 30873 catalog_manager.cc:5495] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e608cb5a7d143c08591c538250f02ce (127.3.236.129). New cstate: current_term: 1 leader_uuid: "8e608cb5a7d143c08591c538250f02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } health_report { overall_health: UNKNOWN } } }
W20260811 07:09:53.807344 31615 tablet.cc:2369] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:09:53.826803 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.824297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.3.236.129:45957)
0811 07:09:53.824349 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 07:09:53.824351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:09:53.824360 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:09:53.824433 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:09:53.824434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:09:53.824455 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:09:53.824517 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:09:53.824670 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.825246 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.826019 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:09:53.826296 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:09:53.826392 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:09:53.826523 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:09:53.826625 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:09:53.826688 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:09:53.826725 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 07:09:53.826744 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:09:53.828724 31547 raft_consensus.cc:1261] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Refusing update from remote peer 8e608cb5a7d143c08591c538250f02ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:53.828728 31331 raft_consensus.cc:1261] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Refusing update from remote peer 8e608cb5a7d143c08591c538250f02ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:09:53.829093 31661 consensus_queue.cc:1035] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:53.829267 31655 consensus_queue.cc:1035] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:09:53.831823 31667 mvcc.cc:205] Tried to move back new op lower bound from 7317226265919393792 to 7317226265770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7317226265919393792}]
I20260811 07:09:53.832095 31675 mvcc.cc:205] Tried to move back new op lower bound from 7317226265919393792 to 7317226265770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7317226265919393792}]
I20260811 07:09:53.832912 31669 mvcc.cc:205] Tried to move back new op lower bound from 7317226265919393792 to 7317226265770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7317226265919393792 or (T in {7317226265919393792})}]
I20260811 07:09:53.834404 30871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:53.834507 30871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:09:53.835294 30871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Sending DeleteTablet for 3 replicas of tablet 7cca963f51ba4900bc4e81e04a76e9eb
I20260811 07:09:53.835644 31510 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55470
I20260811 07:09:53.835688 31298 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:40774
I20260811 07:09:53.835714 31073 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:45134
I20260811 07:09:53.835927 31690 tablet_replica.cc:330] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: stopping tablet replica
I20260811 07:09:53.835958 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 30972
I20260811 07:09:53.836035 31690 raft_consensus.cc:2229] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:53.836143 31690 raft_consensus.cc:2258] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:53.836711 31690 ts_tablet_manager.cc:1905] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:53.837492 31689 tablet_replica.cc:330] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: stopping tablet replica
I20260811 07:09:53.837611 31689 raft_consensus.cc:2229] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:09:53.837730 31689 raft_consensus.cc:2258] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:09:53.838187 31689 ts_tablet_manager.cc:1905] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:09:53.838617 31690 ts_tablet_manager.cc:1918] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:53.838698 31690 log.cc:1196] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/7cca963f51ba4900bc4e81e04a76e9eb
I20260811 07:09:53.838981 31690 ts_tablet_manager.cc:1939] T 7cca963f51ba4900bc4e81e04a76e9eb P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting consensus metadata
I20260811 07:09:53.839443 30854 catalog_manager.cc:4841] TS 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): tablet 7cca963f51ba4900bc4e81e04a76e9eb (table test-table [id=488550653e034f419f92d89b60b1fcc6]) successfully deleted
I20260811 07:09:53.839779 31689 ts_tablet_manager.cc:1918] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:09:53.839869 31689 log.cc:1196] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/7cca963f51ba4900bc4e81e04a76e9eb
I20260811 07:09:53.840346 31689 ts_tablet_manager.cc:1939] T 7cca963f51ba4900bc4e81e04a76e9eb P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting consensus metadata
I20260811 07:09:53.840796 30855 catalog_manager.cc:4841] TS bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): tablet 7cca963f51ba4900bc4e81e04a76e9eb (table test-table [id=488550653e034f419f92d89b60b1fcc6]) successfully deleted
W20260811 07:09:53.842661 30854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:45957 (error 108)
I20260811 07:09:53.842947 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 31192
I20260811 07:09:53.849901 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:09:53.843143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:45957 (local address 127.0.0.1:45138)
0811 07:09:53.849776 (+ 6633us) negotiation.cc:107] Waiting for socket to connect
0811 07:09:53.849849 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6603}
W20260811 07:09:53.855703 30854 catalog_manager.cc:4584] TS 8e608cb5a7d143c08591c538250f02ce (127.3.236.129:45957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cca963f51ba4900bc4e81e04a76e9eb: Network error: Client connection negotiation failed: client connection to 127.3.236.129:45957: connect: Connection refused (error 111)
I20260811 07:09:53.860479 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 31408
I20260811 07:09:53.876690 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 30827
I20260811 07:10:13.892198 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--webserver_interface=127.3.236.190
--webserver_port=46213
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:13.975018 31695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:13.975271 31695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:13.975309 31695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:13.976456 31695 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:10:13.976526 31695 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:10:13.976547 31695 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:10:13.976567 31695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:13.976606 31695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:13.976708 31695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:13.976742 31695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:13.976766 31695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:13.978366 31695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=46213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:13.978682 31695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:13.978930 31695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:13.988549 31710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:13.988549 31707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:13.988615 31705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:13.988958 31695 server_base.cc:1034] running on GCE node
I20260811 07:10:13.989259 31695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:13.989542 31695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:13.990711 31695 hybrid_clock.cc:648] HybridClock initialized: now 1786432213990689 us; error 40 us; skew 500 ppm
Aug 11 07:10:13 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432213, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:13.993378 31695 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:10:13.993821 31695 webserver.cc:460] Webserver started at http://127.3.236.190:46213/ using document root <none> and password file <none>
I20260811 07:10:13.994040 31695 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:13.994095 31695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:13.995447 31695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:13.996088 31720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:13.996282 31695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:13.996374 31695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "d42db9fec6a344b09de9784bfcadd28c"
format_stamp: "Formatted at 2026-08-11 07:09:52 on dist-test-slave-2d19"
server_key: "601e7197b78bba474db727404c59e398"
server_key_iv: "7bf5117ea760a6bda0722ec6e66e576b"
server_key_version: "encryptionkey@0"
I20260811 07:10:13.996767 31695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:14.027310 31695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:14.029917 31695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:14.030052 31695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:14.034166 31695 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35667
I20260811 07:10:14.034277 31812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35667 every 8 connection(s)
I20260811 07:10:14.034655 31695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:14.036310 31816 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:10:14.037040 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap starting.
I20260811 07:10:14.038740 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 31695
I20260811 07:10:14.039052 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:45957
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=38717
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:10:14.046756 31816 log.cc:824] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:14.048938 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:10:14.049270 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap complete.
I20260811 07:10:14.051769 31816 raft_consensus.cc:348] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:14.052057 31816 raft_consensus.cc:729] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Initialized, Role: FOLLOWER
I20260811 07:10:14.052284 31816 consensus_queue.cc:260] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:14.052386 31816 raft_consensus.cc:388] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:14.052448 31816 raft_consensus.cc:482] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:14.052507 31816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:10:14.053586 31816 raft_consensus.cc:504] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:14.053714 31816 leader_election.cc:302] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d42db9fec6a344b09de9784bfcadd28c; no voters:
I20260811 07:10:14.053946 31816 leader_election.cc:288] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:10:14.054024 31821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:10:14.054203 31821 raft_consensus.cc:686] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 LEADER]: Becoming Leader. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Running, Role: LEADER
I20260811 07:10:14.054236 31816 sys_catalog.cc:564] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:14.054308 31821 consensus_queue.cc:237] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:14.054692 31823 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:10:14.054795 31823 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:14.055058 31824 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d42db9fec6a344b09de9784bfcadd28c. Latest consensus state: current_term: 2 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:10:14.055131 31835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:14.055712 31824 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:14.056134 31835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:14.056326 31835 catalog_manager.cc:1259] Loaded cluster ID: 96e521775eac4ac288958f1f85032812
I20260811 07:10:14.056608 31835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:14.057778 31835 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:14.057966 31835 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Loaded TSK: 0
I20260811 07:10:14.068034 31835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Generated new TSK 1
I20260811 07:10:14.068147 31835 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:10:14.126735 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:14.126981 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:14.127019 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:14.128170 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:14.128252 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:14.128319 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:14.129788 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:45957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=38717
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:14.130052 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:14.130285 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:14.136291 31860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.136390 31862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.136456 31859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:14.136596 31819 server_base.cc:1034] running on GCE node
I20260811 07:10:14.136790 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:14.137048 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:14.138232 31819 hybrid_clock.cc:648] HybridClock initialized: now 1786432214138218 us; error 32 us; skew 500 ppm
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:14.141390 31819 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:10:14.141983 31819 webserver.cc:460] Webserver started at http://127.3.236.129:38717/ using document root <none> and password file <none>
I20260811 07:10:14.142246 31819 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:14.142329 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:14.144155 31819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:10:14.145036 31875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.145280 31819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:14.145381 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "8e608cb5a7d143c08591c538250f02ce"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "0139a370a7d840afea486c163ab17bd1"
server_key_iv: "cf798ec1dd022664c2efaaea4413f676"
server_key_version: "encryptionkey@0"
I20260811 07:10:14.145835 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:14.157877 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:14.160569 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:14.160719 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:14.160980 31819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:14.161502 31887 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:10:14.163012 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:10:14.163076 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 07:10:14.163110 31819 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:10:14.164028 31819 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:10:14.164075 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 07:10:14.164165 31887 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Bootstrap starting.
I20260811 07:10:14.171689 31819 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:45957
I20260811 07:10:14.171957 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:45957 every 8 connection(s)
I20260811 07:10:14.172158 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:14.175032 31887 log.cc:824] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:14.176098 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 31819
I20260811 07:10:14.176452 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:44809
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=40549
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:10:14.177810 31887 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:10:14.178083 31887 tablet_bootstrap.cc:492] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Bootstrap complete.
I20260811 07:10:14.179411 31887 ts_tablet_manager.cc:1397] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260811 07:10:14.180845 31887 raft_consensus.cc:348] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:10:14.181108 31887 raft_consensus.cc:729] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Initialized, Role: FOLLOWER
I20260811 07:10:14.181272 31887 consensus_queue.cc:260] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } }
I20260811 07:10:14.181561 31887 ts_tablet_manager.cc:1428] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:14.190193 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.172926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:57477)
0811 07:10:14.175244 (+ 2318us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.175296 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.176010 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.176972 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.176976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.178617 (+ 1641us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:14.178904 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.178911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.180681 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.180683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.181871 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.181874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.182721 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.182723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.183058 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.183736 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:14.183748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:14.185664 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:14.188226 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.188232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.188235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.188568 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.188887 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.188888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.188889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.189021 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.189529 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:14.189536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:14.189694 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.189807 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.189972 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2261,"thread_start_us":122,"threads_started":1}
I20260811 07:10:14.190665 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.173018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:57477 (local address 127.3.236.190:35667)
0811 07:10:14.175438 (+ 2420us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.175447 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.175507 (+ 60us) server_negotiation.cc:407] Connection header received
0811 07:10:14.176395 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.176405 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.176468 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.176813 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:14.179059 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.180558 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.181983 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.182610 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.182737 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.185806 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:14.185811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:14.185811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:14.185825 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:14.188119 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.188666 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.188670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.188670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.188802 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.189103 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.189104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.189104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.189440 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:14.189554 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.189900 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.190059 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2370,"thread_start_us":84,"threads_started":1}
I20260811 07:10:14.191642 32071 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
I20260811 07:10:14.191746 32071 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:14.191938 32071 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:10:14.192776 31743 ts_manager.cc:194] Registered new tserver with Master: 8e608cb5a7d143c08591c538250f02ce (127.3.236.129:45957)
I20260811 07:10:14.193970 31743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:57477
I20260811 07:10:14.194924 32071 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
I20260811 07:10:14.199205 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.194285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:45957 (local address 127.0.0.1:53928)
0811 07:10:14.194912 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.194951 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.195086 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.195408 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.195411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.195413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:14.195486 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.195487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.196757 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.196758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.198020 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.198022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.198915 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.198916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.198999 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.199012 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.199082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.199130 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":113,"threads_started":1}
I20260811 07:10:14.199215 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.194625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.3.236.129:45957)
0811 07:10:14.195220 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.195225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.195233 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:10:14.195266 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.195269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.195283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.195348 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:14.195567 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.196657 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.198111 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.198824 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.198903 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.198954 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.199109 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.199147 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":94,"threads_started":1}
I20260811 07:10:14.200090 31953 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cca963f51ba4900bc4e81e04a76e9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:09:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:53928
I20260811 07:10:14.203359 32087 tablet_replica.cc:330] stopping tablet replica
I20260811 07:10:14.203498 32087 raft_consensus.cc:2229] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:14.203583 32087 raft_consensus.cc:2258] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:14.203894 32087 ts_tablet_manager.cc:1905] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:14.206281 32087 ts_tablet_manager.cc:1918] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:14.206372 32087 log.cc:1196] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal/wals/7cca963f51ba4900bc4e81e04a76e9eb
I20260811 07:10:14.206625 32087 ts_tablet_manager.cc:1939] T 7cca963f51ba4900bc4e81e04a76e9eb P 8e608cb5a7d143c08591c538250f02ce: Deleting consensus metadata
I20260811 07:10:14.207140 31723 catalog_manager.cc:4841] TS 8e608cb5a7d143c08591c538250f02ce (127.3.236.129:45957): tablet 7cca963f51ba4900bc4e81e04a76e9eb (table test-table [id=488550653e034f419f92d89b60b1fcc6]) successfully deleted
W20260811 07:10:14.267827 32078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:14.268033 32078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:14.268056 32078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:14.269238 32078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:14.269281 32078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:14.269307 32078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:14.270757 32078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:44809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=40549
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:14.271013 32078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:14.271256 32078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:14.276471 32097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.276466 32093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.276450 32094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:14.276559 32078 server_base.cc:1034] running on GCE node
I20260811 07:10:14.276873 32078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:14.277125 32078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:14.278290 32078 hybrid_clock.cc:648] HybridClock initialized: now 1786432214278275 us; error 28 us; skew 500 ppm
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:14.280858 32078 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:10:14.281301 32078 webserver.cc:460] Webserver started at http://127.3.236.130:40549/ using document root <none> and password file <none>
I20260811 07:10:14.281558 32078 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:14.281644 32078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:14.283113 32078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:14.283838 32104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.284101 32078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:14.284210 32078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "14720846d6472e0b9ef38f99122ca7f1"
server_key_iv: "0ce306fb2750de8f3e3aa472d8dc278f"
server_key_version: "encryptionkey@0"
I20260811 07:10:14.284622 32078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:14.321302 32078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:14.323566 32078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:14.323753 32078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:14.324056 32078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:14.324419 32078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:14.324476 32078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.324514 32078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:14.324555 32078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.330698 32078 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:44809
I20260811 07:10:14.330750 32289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:44809 every 8 connection(s)
I20260811 07:10:14.331132 32078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:14.334542 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 32078
I20260811 07:10:14.334846 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:38059
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=37399
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:14.342267 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.331888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:35947 (local address 127.3.236.190:35667)
0811 07:10:14.331958 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.331960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.332239 (+ 279us) server_negotiation.cc:407] Connection header received
0811 07:10:14.332754 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.332755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.332767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.332820 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:14.333771 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.334269 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.335812 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.336017 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.336077 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.338745 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:14.338747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:14.338748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:14.338755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:14.340717 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.341146 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.341148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.341148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.341216 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.341473 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.341481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.341481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.341658 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:14.341738 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.341988 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.342122 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:10:14.342283 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.331910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:35947)
0811 07:10:14.332103 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.332141 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.332661 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.332906 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.332910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.333488 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:14.333685 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.333690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.334367 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.334369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.335715 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.335719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.336110 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.336111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.336310 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.336933 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:14.336943 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:14.338642 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:14.340813 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.340820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.340822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.341064 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.341291 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.341292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.341292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.341403 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.341732 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:14.341737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:14.341854 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.341943 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.342069 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260811 07:10:14.342857 32290 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
I20260811 07:10:14.342959 32290 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:14.343191 32290 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:10:14.343659 31743 ts_manager.cc:194] Registered new tserver with Master: 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809)
I20260811 07:10:14.344085 31743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:35947
W20260811 07:10:14.424082 32297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:14.424357 32297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:14.424402 32297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:14.425664 32297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:14.425730 32297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:14.425776 32297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:14.427307 32297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:38059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=37399
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:14.427595 32297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:14.427855 32297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:14.433274 32304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.433288 32306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:14.433288 32303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:14.433495 32297 server_base.cc:1034] running on GCE node
I20260811 07:10:14.433743 32297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:14.434002 32297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:14.435150 32297 hybrid_clock.cc:648] HybridClock initialized: now 1786432214435127 us; error 37 us; skew 500 ppm
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:14.437783 32297 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:10:14.438215 32297 webserver.cc:460] Webserver started at http://127.3.236.131:37399/ using document root <none> and password file <none>
I20260811 07:10:14.438431 32297 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:14.438488 32297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:14.439872 32297 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:14.440555 32321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.440732 32297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:14.440821 32297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "4cf64a23ed0fd1d127191b24250fa220"
server_key_iv: "544e18d1063f64f77e6fa58f07a2629e"
server_key_version: "encryptionkey@0"
I20260811 07:10:14.441164 32297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:14.453778 32297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:14.456375 32297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:14.456507 32297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:14.456724 32297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:14.457063 32297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:14.457099 32297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.457155 32297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:14.457182 32297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.463382 32297 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:38059
I20260811 07:10:14.463503 32516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:38059 every 8 connection(s)
I20260811 07:10:14.463836 32297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:10:14 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432214, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:14.472002 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 32297
I20260811 07:10:14.473722 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.464635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:60385 (local address 127.3.236.190:35667)
0811 07:10:14.464686 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.464688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.464969 (+ 281us) server_negotiation.cc:407] Connection header received
0811 07:10:14.465490 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.465492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.465502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.465553 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:14.466588 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.467069 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.468103 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.468350 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.468411 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.470631 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:14.470633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:14.470633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:14.470640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:14.471913 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.472558 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.472560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.472560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.472639 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.473015 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.473016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.473017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.473150 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:14.473229 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.473587 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.473656 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:10:14.474002 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.464620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:60385)
0811 07:10:14.464785 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.464849 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.465396 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.465642 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.465646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.466272 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:14.466503 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.466509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.467153 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.467154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.468028 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.468030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.468422 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.468423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.468550 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.469034 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:14.469043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:14.470541 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:14.472022 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.472030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.472033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.472416 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.472759 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:14.472761 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:14.472761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:14.472921 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:14.473234 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:14.473242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:14.473416 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.473554 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.473734 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260811 07:10:14.474521 32521 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35667
I20260811 07:10:14.474632 32521 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:14.474843 32521 heartbeater.cc:511] Master 127.3.236.190:35667 requested a full tablet report, sending...
I20260811 07:10:14.475279 31743 ts_manager.cc:194] Registered new tserver with Master: bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:10:14.475646 31743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:60385
I20260811 07:10:14.479558 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.472499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.3.236.190:35667)
0811 07:10:14.472618 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.472620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.472732 (+ 112us) server_negotiation.cc:407] Connection header received
0811 07:10:14.472761 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.472761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.472774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.472840 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:14.473408 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.474224 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.475276 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.475561 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.475621 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.476962 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:14.476964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:14.476964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:14.476971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:14.478615 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.478933 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.478934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.478934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.479016 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.479171 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.479172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.479172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.479268 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:14.479390 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.479419 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.479487 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260811 07:10:14.480363 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:14.483040 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.480725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.3.236.190:35667)
0811 07:10:14.480785 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.480787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.480866 (+ 79us) server_negotiation.cc:407] Connection header received
0811 07:10:14.480923 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.480923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.480952 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.481017 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:14.481104 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.481597 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.482375 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.482582 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.482636 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.482762 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:14.482804 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 07:10:14.482994 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42220: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 07:10:14.483114 32527 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42220: token expired
W20260811 07:10:14.484081 31625 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.236.190:35667)
I20260811 07:10:14.490295 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.484380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.3.236.190:35667)
0811 07:10:14.484417 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.484419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.484428 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:14.484498 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.484498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.484506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.484561 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:14.485007 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.485469 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.486193 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.486386 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.486444 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.487727 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:14.487728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:14.487729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:14.487734 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:14.489202 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.489553 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.489553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.489554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.489608 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:14.489797 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:14.489797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:14.489798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:14.489926 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:14.490008 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.490163 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.490230 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:10:14.491535 31743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:14.493533 31743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:14.495750 31953 tablet_service.cc:1467] Processing CreateTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b (DEFAULT_TABLE table=test-table [id=d5d48a2ad06a49d390e67eb16e69e2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:14.495954 31953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37575fb8352b40b89718ab8faaf1cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:14.497437 31887 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Bootstrap starting.
I20260811 07:10:14.497776 31887 tablet_bootstrap.cc:654] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:14.498420 31887 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: No bootstrap required, opened a new log
I20260811 07:10:14.498474 31887 ts_tablet_manager.cc:1397] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:10:14.498584 31887 raft_consensus.cc:348] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.498642 31887 raft_consensus.cc:374] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:14.498684 31887 raft_consensus.cc:729] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Initialized, Role: FOLLOWER
I20260811 07:10:14.498730 31887 consensus_queue.cc:260] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.498804 31887 ts_tablet_manager.cc:1428] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:14.500249 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.495441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:38059 (local address 127.0.0.1:44666)
0811 07:10:14.495620 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.495631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.495674 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.496165 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.496166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.496169 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:14.496217 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.496219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.497550 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.497551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.498639 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.498642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.500015 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.500016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.500069 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.500073 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.500121 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.500156 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":132,"threads_started":1}
I20260811 07:10:14.500317 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.495731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.3.236.131:38059)
0811 07:10:14.495938 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.495943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.495956 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:10:14.496023 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.496025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.496044 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.496165 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:14.496334 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.497411 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.498770 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.499926 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.500009 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.500071 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.500136 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.500188 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260811 07:10:14.501169 32406 tablet_service.cc:1467] Processing CreateTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b (DEFAULT_TABLE table=test-table [id=d5d48a2ad06a49d390e67eb16e69e2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:14.501544 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.495393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44809 (local address 127.0.0.1:34208)
0811 07:10:14.495438 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.495451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.495506 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.497735 (+ 2229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.497736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.497738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:14.497772 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.497773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.498993 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.498995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.500134 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.500138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.501264 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.501265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.501334 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.501338 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.501411 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.501457 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 07:10:14.501569 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.495503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.3.236.130:44809)
0811 07:10:14.497548 (+ 2045us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.497553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.497569 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:10:14.497613 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.497615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.497631 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.497726 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:14.497829 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.498881 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.500222 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.501170 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.501234 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.501275 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.501455 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.501497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2002,"thread_start_us":114,"threads_started":1}
I20260811 07:10:14.501610 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37575fb8352b40b89718ab8faaf1cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:14.502238 32183 tablet_service.cc:1467] Processing CreateTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b (DEFAULT_TABLE table=test-table [id=d5d48a2ad06a49d390e67eb16e69e2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:14.502559 32183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37575fb8352b40b89718ab8faaf1cd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:14.505106 32539 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Bootstrap starting.
I20260811 07:10:14.505537 32540 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Bootstrap starting.
I20260811 07:10:14.505867 32539 tablet_bootstrap.cc:654] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:14.506330 32540 tablet_bootstrap.cc:654] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:14.506496 32539 log.cc:824] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:14.507016 32540 log.cc:824] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:14.507422 32539 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: No bootstrap required, opened a new log
I20260811 07:10:14.507510 32539 ts_tablet_manager.cc:1397] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:14.507926 32540 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: No bootstrap required, opened a new log
I20260811 07:10:14.508065 32540 ts_tablet_manager.cc:1397] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:10:14.509011 32539 raft_consensus.cc:348] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.509186 32539 raft_consensus.cc:374] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:14.509225 32539 raft_consensus.cc:729] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3c3dd6268246a2bc14b4078f4ad52c, State: Initialized, Role: FOLLOWER
I20260811 07:10:14.509356 32539 consensus_queue.cc:260] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.509657 32539 ts_tablet_manager.cc:1428] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:14.509706 32521 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
I20260811 07:10:14.509953 32540 raft_consensus.cc:348] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.510162 32540 raft_consensus.cc:374] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:14.510217 32540 raft_consensus.cc:729] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, State: Initialized, Role: FOLLOWER
I20260811 07:10:14.510349 32540 consensus_queue.cc:260] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.510676 32540 ts_tablet_manager.cc:1428] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:14.510792 32290 heartbeater.cc:503] Master 127.3.236.190:35667 was elected leader, sending a full tablet report...
W20260811 07:10:14.581700 32294 tablet.cc:2369] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:10:14.675743 32073 tablet.cc:2369] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:10:14.709539 32082 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:14.709710 32082 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.710129 32082 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:10:14.713163 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.710431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:34707 (local address 127.3.236.130:44809)
0811 07:10:14.710507 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.710510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.710574 (+ 64us) server_negotiation.cc:407] Connection header received
0811 07:10:14.710651 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.710652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.710668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.710734 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:14.710888 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.711361 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712447 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.712813 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712899 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.712908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.713046 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.713082 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:10:14.713179 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.710451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:49513 (local address 127.3.236.131:38059)
0811 07:10:14.710505 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.710507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.710625 (+ 118us) server_negotiation.cc:407] Connection header received
0811 07:10:14.710715 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.710716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.710725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.710800 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:14.710943 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.711422 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712446 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.712821 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712908 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.712917 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.713068 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.713107 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:10:14.713201 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.710325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:38059 (local address 127.3.236.129:49513)
0811 07:10:14.710535 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.710545 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.710644 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.710803 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.710804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.710806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:14.710865 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.710867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.711512 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.711513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.712357 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.712360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.712908 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.712971 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.712975 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.713058 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.713099 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":14,"thread_start_us":50,"threads_started":1}
I20260811 07:10:14.713271 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.710326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44809 (local address 127.3.236.129:34707)
0811 07:10:14.710493 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.710505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:14.710582 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:14.710739 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:14.710740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:14.710742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:14.710807 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.710809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.711455 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.711456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.712351 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:14.712354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.712899 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.712900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:14.712962 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.712966 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:14.713041 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:10:14.713084 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":44,"thread_start_us":60,"threads_started":1}
I20260811 07:10:14.713621 32213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37575fb8352b40b89718ab8faaf1cd7b" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" is_pre_election: true
I20260811 07:10:14.713706 32439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37575fb8352b40b89718ab8faaf1cd7b" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" is_pre_election: true
I20260811 07:10:14.713793 32213 raft_consensus.cc:2454] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 0.
I20260811 07:10:14.713824 32439 raft_consensus.cc:2454] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 0.
I20260811 07:10:14.714039 31881 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, 8e608cb5a7d143c08591c538250f02ce; no voters:
I20260811 07:10:14.714188 32082 raft_consensus.cc:2781] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:14.714257 32082 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:14.714274 32082 raft_consensus.cc:3037] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 0 FOLLOWER]: Advancing to term 1
W20260811 07:10:14.714413 32522 tablet.cc:2369] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:10:14.715058 32082 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.715194 32082 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 election: Requested vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
I20260811 07:10:14.715349 32439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37575fb8352b40b89718ab8faaf1cd7b" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
I20260811 07:10:14.715360 32213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37575fb8352b40b89718ab8faaf1cd7b" candidate_uuid: "8e608cb5a7d143c08591c538250f02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
I20260811 07:10:14.715427 32439 raft_consensus.cc:3037] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:14.715440 32213 raft_consensus.cc:3037] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:14.716187 32213 raft_consensus.cc:2454] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 1.
I20260811 07:10:14.716197 32439 raft_consensus.cc:2454] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e608cb5a7d143c08591c538250f02ce in term 1.
I20260811 07:10:14.716378 31881 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, 8e608cb5a7d143c08591c538250f02ce; no voters:
I20260811 07:10:14.716467 32082 raft_consensus.cc:2781] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:14.716619 32082 raft_consensus.cc:686] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 LEADER]: Becoming Leader. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Running, Role: LEADER
I20260811 07:10:14.716691 32082 consensus_queue.cc:237] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:14.717430 31743 catalog_manager.cc:5495] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e608cb5a7d143c08591c538250f02ce (127.3.236.129). New cstate: current_term: 1 leader_uuid: "8e608cb5a7d143c08591c538250f02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } health_report { overall_health: HEALTHY } } }
I20260811 07:10:14.727815 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.725030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.3.236.129:45957)
0811 07:10:14.725210 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:14.725212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:14.725218 (+ 6us) server_negotiation.cc:407] Connection header received
0811 07:10:14.725249 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:14.725249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:14.725265 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:14.725350 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:14.725500 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.725963 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.726708 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:14.726919 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:14.727009 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:14.727450 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:14.727554 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:14.727641 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:14.727685 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 07:10:14.727704 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260811 07:10:14.729460 32213 raft_consensus.cc:1261] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Refusing update from remote peer 8e608cb5a7d143c08591c538250f02ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:14.729456 32439 raft_consensus.cc:1261] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Refusing update from remote peer 8e608cb5a7d143c08591c538250f02ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:14.729846 32551 consensus_queue.cc:1035] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:14.729955 32082 consensus_queue.cc:1035] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:14.732312 32557 mvcc.cc:205] Tried to move back new op lower bound from 7317226351529058304 to 7317226351480229888. Current Snapshot: MvccSnapshot[applied={T|T < 7317226351529058304}]
I20260811 07:10:14.733412 32558 mvcc.cc:205] Tried to move back new op lower bound from 7317226351529058304 to 7317226351480229888. Current Snapshot: MvccSnapshot[applied={T|T < 7317226351529058304 or (T in {7317226351529058304})}]
I20260811 07:10:14.735026 31742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:14.735136 31742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:14.735930 31742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Sending DeleteTablet for 3 replicas of tablet 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:14.736328 31953 tablet_service.cc:1514] Processing DeleteTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:14 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:53928
I20260811 07:10:14.736342 32406 tablet_service.cc:1514] Processing DeleteTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:14 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:44666
I20260811 07:10:14.736440 32183 tablet_service.cc:1514] Processing DeleteTablet for tablet 37575fb8352b40b89718ab8faaf1cd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:14 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:34208
I20260811 07:10:14.736552 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 31819
I20260811 07:10:14.753000 32572 tablet_replica.cc:330] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: stopping tablet replica
I20260811 07:10:14.753273 32572 raft_consensus.cc:2229] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:14.753590 32572 raft_consensus.cc:2258] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:14.755874 32572 ts_tablet_manager.cc:1905] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:14.756150 32574 tablet_replica.cc:330] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: stopping tablet replica
I20260811 07:10:14.756345 32574 raft_consensus.cc:2229] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:14.756506 32574 raft_consensus.cc:2258] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:14.756963 32574 ts_tablet_manager.cc:1905] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:14.758121 32572 ts_tablet_manager.cc:1918] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:14.758215 32572 log.cc:1196] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:14.758424 32574 ts_tablet_manager.cc:1918] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:14.758531 32574 log.cc:1196] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:14.758546 32572 ts_tablet_manager.cc:1939] T 37575fb8352b40b89718ab8faaf1cd7b P bd3c3dd6268246a2bc14b4078f4ad52c: Deleting consensus metadata
I20260811 07:10:14.758821 32574 ts_tablet_manager.cc:1939] T 37575fb8352b40b89718ab8faaf1cd7b P 3c09ecebfc8b4e28ae8be7e69b7e9fdd: Deleting consensus metadata
I20260811 07:10:14.759066 31724 catalog_manager.cc:4841] TS bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): tablet 37575fb8352b40b89718ab8faaf1cd7b (table test-table [id=d5d48a2ad06a49d390e67eb16e69e2d2]) successfully deleted
I20260811 07:10:14.759299 31723 catalog_manager.cc:4841] TS 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): tablet 37575fb8352b40b89718ab8faaf1cd7b (table test-table [id=d5d48a2ad06a49d390e67eb16e69e2d2]) successfully deleted
W20260811 07:10:14.761122 31723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:45957 (error 108)
I20260811 07:10:14.761467 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 32078
I20260811 07:10:14.761688 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:14.761499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:45957 (local address 127.0.0.1:53940)
0811 07:10:14.761577 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:14.761644 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 07:10:14.763921 31723 catalog_manager.cc:4584] TS 8e608cb5a7d143c08591c538250f02ce (127.3.236.129:45957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37575fb8352b40b89718ab8faaf1cd7b: Network error: Client connection negotiation failed: client connection to 127.3.236.129:45957: connect: Connection refused (error 111)
I20260811 07:10:14.775658 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 32297
I20260811 07:10:14.789240 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 31695
I20260811 07:10:34.809940 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--webserver_interface=127.3.236.190
--webserver_port=46213
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:34.896775 32580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:34.896989 32580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:34.897011 32580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:34.898074 32580 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:10:34.898108 32580 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:10:34.898121 32580 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:10:34.898132 32580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:34.898145 32580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:34.898159 32580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:34.898186 32580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:34.898198 32580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:34.899633 32580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=46213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:34.899835 32580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:34.900009 32580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:34.905400 32591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:34.905412 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:34.905510 32580 server_base.cc:1034] running on GCE node
W20260811 07:10:34.905400 32593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:34.905988 32580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:34.906241 32580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:34.907393 32580 hybrid_clock.cc:648] HybridClock initialized: now 1786432234907365 us; error 52 us; skew 500 ppm
Aug 11 07:10:34 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432234, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:34.910270 32580 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:10:34.910683 32580 webserver.cc:460] Webserver started at http://127.3.236.190:46213/ using document root <none> and password file <none>
I20260811 07:10:34.910861 32580 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:34.910904 32580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:34.912237 32580 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:34.913028 32601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:34.913219 32580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:34.913329 32580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "d42db9fec6a344b09de9784bfcadd28c"
format_stamp: "Formatted at 2026-08-11 07:09:52 on dist-test-slave-2d19"
server_key: "601e7197b78bba474db727404c59e398"
server_key_iv: "7bf5117ea760a6bda0722ec6e66e576b"
server_key_version: "encryptionkey@0"
I20260811 07:10:34.913689 32580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:34.923684 32580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:34.925747 32580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:34.925913 32580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:34.930215 32580 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35667
I20260811 07:10:34.930254 32705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35667 every 8 connection(s)
I20260811 07:10:34.930648 32580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:34.932217 32706 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:10:34.932960 32706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap starting.
I20260811 07:10:34.937081 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 32580
I20260811 07:10:34.937424 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:45957
--local_ip_for_outbound_sockets=127.3.236.129
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=38717
--webserver_interface=127.3.236.129
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:10:34.943080 32706 log.cc:824] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:34.945546 32706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:10:34.945883 32706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Bootstrap complete.
I20260811 07:10:34.948246 32706 raft_consensus.cc:348] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:34.948575 32706 raft_consensus.cc:729] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Initialized, Role: FOLLOWER
I20260811 07:10:34.948755 32706 consensus_queue.cc:260] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:34.948817 32706 raft_consensus.cc:388] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:34.948882 32706 raft_consensus.cc:482] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:34.948937 32706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 2 FOLLOWER]: Advancing to term 3
I20260811 07:10:34.950001 32706 raft_consensus.cc:504] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:34.950120 32706 leader_election.cc:302] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d42db9fec6a344b09de9784bfcadd28c; no voters:
I20260811 07:10:34.950336 32706 leader_election.cc:288] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 07:10:34.950376 32711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 3 FOLLOWER]: Leader election won for term 3
I20260811 07:10:34.950505 32711 raft_consensus.cc:686] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [term 3 LEADER]: Becoming Leader. State: Replica: d42db9fec6a344b09de9784bfcadd28c, State: Running, Role: LEADER
I20260811 07:10:34.950596 32711 consensus_queue.cc:237] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } }
I20260811 07:10:34.950644 32706 sys_catalog.cc:564] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:34.950862 32711 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d42db9fec6a344b09de9784bfcadd28c. Latest consensus state: current_term: 3 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:10:34.951051 32711 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:34.950861 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d42db9fec6a344b09de9784bfcadd28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d42db9fec6a344b09de9784bfcadd28c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35667 } } }
I20260811 07:10:34.951363 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:34.951514 32725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:34.952370 32725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:34.952476 32725 catalog_manager.cc:1259] Loaded cluster ID: 96e521775eac4ac288958f1f85032812
I20260811 07:10:34.952603 32725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:34.953958 32725 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:34.954154 32725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Loaded TSK: 0
I20260811 07:10:34.954207 32725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d42db9fec6a344b09de9784bfcadd28c: Loaded TSK: 1
F20260811 07:10:34.954432 32725 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7eff9e7fc9ad google::LogMessage::Fail() at ??:0
@ 0x7eff9e7ff88a google::LogMessage::SendToLog() at ??:0
@ 0x7eff9e7fc4de google::LogMessage::Flush() at ??:0
@ 0x7eff9e7fda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7effa0f69104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7effa0f66d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7effa67561cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7effa675648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7effa6756594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7effa671cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7effa6726ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7effa06ca0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7effa06bf954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7effa15236db start_thread at ??:0
@ 0x7eff9d2fa61f clone at ??:0
W20260811 07:10:35.050143 32709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:35.050352 32709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:35.050376 32709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:35.051529 32709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:35.051579 32709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:35.051607 32709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:35.053175 32709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:45957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=38717
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:35.053431 32709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:35.053689 32709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:35.058934 308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:35.059110 307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:35.058991 32709 server_base.cc:1034] running on GCE node
W20260811 07:10:35.058938 310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:35.059423 32709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:35.059723 32709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:35.060879 32709 hybrid_clock.cc:648] HybridClock initialized: now 1786432235060861 us; error 34 us; skew 500 ppm
Aug 11 07:10:35 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:35.063649 32709 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:10:35.064136 32709 webserver.cc:460] Webserver started at http://127.3.236.129:38717/ using document root <none> and password file <none>
I20260811 07:10:35.064416 32709 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:35.064488 32709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:35.066030 32709 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:35.066779 324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.066954 32709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:35.067061 32709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "8e608cb5a7d143c08591c538250f02ce"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "0139a370a7d840afea486c163ab17bd1"
server_key_iv: "cf798ec1dd022664c2efaaea4413f676"
server_key_version: "encryptionkey@0"
I20260811 07:10:35.067432 32709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:35.094833 32709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:35.097363 32709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:35.097545 32709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:35.097810 32709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:35.098292 336 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:10:35.100324 32709 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:10:35.100392 32709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 07:10:35.100456 32709 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:10:35.101142 32709 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:10:35.101202 32709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 07:10:35.101282 336 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Bootstrap starting.
I20260811 07:10:35.108891 32709 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:45957
I20260811 07:10:35.109268 32709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:35.109731 512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:45957 every 8 connection(s)
I20260811 07:10:35.113052 336 log.cc:824] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:35.114594 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 32709
I20260811 07:10:35.114892 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:44809
--local_ip_for_outbound_sockets=127.3.236.130
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=40549
--webserver_interface=127.3.236.130
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:10:35.115846 336 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:10:35.116176 336 tablet_bootstrap.cc:492] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Bootstrap complete.
I20260811 07:10:35.117550 336 ts_tablet_manager.cc:1397] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.011s
I20260811 07:10:35.119005 336 raft_consensus.cc:348] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:35.119338 336 raft_consensus.cc:729] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e608cb5a7d143c08591c538250f02ce, State: Initialized, Role: FOLLOWER
I20260811 07:10:35.119515 336 consensus_queue.cc:260] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:35.119809 336 ts_tablet_manager.cc:1428] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
W20260811 07:10:35.167008 516 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:10:35.109999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:39409)
0811 07:10:35.110231 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.110273 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:35.110733 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:35.166805 (+ 56072us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
W20260811 07:10:35.167294 514 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:35667 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:35667: Client connection negotiation failed: client connection to 127.3.236.190:35667: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:10:35.167665 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.167539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:60203)
0811 07:10:35.167595 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.167634 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:10:35.168061 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.167972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:35687)
0811 07:10:35.168021 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.168045 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:10:35.168187 514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:10:35.206091 518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:35.206334 518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:35.206372 518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:35.207515 518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:35.207578 518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:35.207624 518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:35.209156 518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:44809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=40549
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:35.209444 518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:35.209681 518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:35.215039 536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:35.215046 529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:35.215080 531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:35.215305 518 server_base.cc:1034] running on GCE node
I20260811 07:10:35.215514 518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:35.215747 518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:35.216910 518 hybrid_clock.cc:648] HybridClock initialized: now 1786432235216896 us; error 28 us; skew 500 ppm
Aug 11 07:10:35 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:35.219424 518 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:10:35.219856 518 webserver.cc:460] Webserver started at http://127.3.236.130:40549/ using document root <none> and password file <none>
I20260811 07:10:35.220077 518 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:35.220142 518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:35.221522 518 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:35.222150 547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.222340 518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:35.222438 518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "14720846d6472e0b9ef38f99122ca7f1"
server_key_iv: "0ce306fb2750de8f3e3aa472d8dc278f"
server_key_version: "encryptionkey@0"
I20260811 07:10:35.222841 518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:35.237528 518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:35.240198 518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:35.240407 518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:35.240718 518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:35.241078 518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:35.241128 518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.241168 518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:35.241202 518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.248342 518 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:44809
I20260811 07:10:35.248431 746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:44809 every 8 connection(s)
I20260811 07:10:35.248756 518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:35.249768 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.249393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:46805)
0811 07:10:35.249550 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.249622 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260811 07:10:35.250236 747 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:35667 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:35667: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
I20260811 07:10:35.250535 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.250430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:50399)
0811 07:10:35.250489 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.250517 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 07:10:35.250797 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.250742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:36403)
0811 07:10:35.250766 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.250785 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260811 07:10:35.250908 747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:10:35.256719 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 518
I20260811 07:10:35.257084 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:38059
--local_ip_for_outbound_sockets=127.3.236.131
--tserver_master_addrs=127.3.236.190:35667
--webserver_port=37399
--webserver_interface=127.3.236.131
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:35.346601 751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:35.346796 751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:35.346819 751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:35.348078 751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:35.348124 751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:35.348150 751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:35.349538 751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:38059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=37399
--tserver_master_addrs=127.3.236.190:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:35.349732 751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:35.349928 751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:10:35.355077 751 server_base.cc:1034] running on GCE node
W20260811 07:10:35.355306 767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:35.355324 763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:35.355314 769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:35.355696 751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:35.355981 751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:35.357179 751 hybrid_clock.cc:648] HybridClock initialized: now 1786432235357171 us; error 36 us; skew 500 ppm
Aug 11 07:10:35 dist-test-slave-2d19 krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:35.359848 751 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:10:35.360491 751 webserver.cc:460] Webserver started at http://127.3.236.131:37399/ using document root <none> and password file <none>
I20260811 07:10:35.360765 751 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:35.360845 751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:35.362324 751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:10:35.362977 780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.363184 751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:35.363283 751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "bd3c3dd6268246a2bc14b4078f4ad52c"
format_stamp: "Formatted at 2026-08-11 07:09:53 on dist-test-slave-2d19"
server_key: "4cf64a23ed0fd1d127191b24250fa220"
server_key_iv: "544e18d1063f64f77e6fa58f07a2629e"
server_key_version: "encryptionkey@0"
I20260811 07:10:35.363621 751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:35.384436 751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:35.386484 751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:35.386611 751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:35.386819 751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:35.387226 751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:35.387265 751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.387315 751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:35.387347 751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:35.394423 751 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:38059
I20260811 07:10:35.394493 975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:38059 every 8 connection(s)
I20260811 07:10:35.394858 751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:10:35.396087 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.395589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:54811)
0811 07:10:35.395758 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.395882 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
W20260811 07:10:35.396553 978 heartbeater.cc:644] Failed to heartbeat to 127.3.236.190:35667 (0 consecutive failures): Network error: Failed to ping master at 127.3.236.190:35667: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
I20260811 07:10:35.396852 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.396752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:46971)
0811 07:10:35.396801 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.396831 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:10:35.397145 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:35.397088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:50475)
0811 07:10:35.397110 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:35.397131 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260811 07:10:35.397258 978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:10:35.403625 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 751
I20260811 07:10:36.168777 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.168533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:52969)
0811 07:10:36.168669 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:36.168710 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260811 07:10:36.251504 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.251301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:51549)
0811 07:10:36.251418 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:36.251454 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260811 07:10:36.397837 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.397618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:56811)
0811 07:10:36.397744 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:36.397781 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260811 07:10:36.847189 986 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:36.847317 986 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:36.847607 986 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
Aug 11 07:10:36 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.130@KRBTEST.COM
Aug 11 07:10:36 dist-test-slave-2d19 krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.131@KRBTEST.COM
I20260811 07:10:36.857969 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.847772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44809 (local address 127.3.236.129:34241)
0811 07:10:36.847909 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:36.847921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:36.847983 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:36.848874 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:36.848876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:36.849293 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:36.849494 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:36.849501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.850605 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.850606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:36.851368 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:36.851371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.851980 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.851981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:36.852110 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:36.852769 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:36.852781 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:36.854249 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:36.856428 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:36.856436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:36.856438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:36.856664 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:36.856936 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:36.856937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:36.856938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:36.857051 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:36.857437 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:36.857444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:36.857582 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 07:10:36.857695 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:10:36.857862 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":59,"thread_start_us":74,"threads_started":1}
I20260811 07:10:36.857980 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.848011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:34241 (local address 127.3.236.130:44809)
0811 07:10:36.848170 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:36.848178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:36.848195 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:10:36.848667 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:36.848670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:36.848719 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:36.848864 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:36.849631 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.850481 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.851461 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.851853 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.852008 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:36.854364 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:36.854368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:36.854369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:36.854383 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:36.856324 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:36.856753 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:36.856757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:36.856757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:36.856858 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:36.857122 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:36.857123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:36.857123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:36.857362 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:36.857456 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:36.857751 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 07:10:36.857872 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260811 07:10:36.858264 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.847907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:54905 (local address 127.3.236.131:38059)
0811 07:10:36.848093 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:36.848106 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:36.848146 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:10:36.848625 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:36.848628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:36.848672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:36.848833 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:36.849628 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.850496 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.851455 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.851853 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.852019 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:36.854851 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:36.854854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:36.854854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:36.854864 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:36.856719 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:36.857158 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:36.857161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:36.857161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:36.857255 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:36.857466 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:36.857468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:36.857468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:36.857700 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:36.857787 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:36.857983 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 07:10:36.858126 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
W20260811 07:10:36.858763 332 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:36.858826 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:36.847772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:38059 (local address 127.3.236.129:54905)
0811 07:10:36.847932 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:36.847946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:36.848030 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:36.848872 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:36.848875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:36.849293 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:36.849494 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:36.849501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.850601 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.850602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:36.851367 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:36.851369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:36.851992 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:36.851993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:36.852112 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:36.852729 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:36.852737 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:36.854736 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:36.856824 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:36.856826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:36.856826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:36.857060 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:36.857328 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:36.857328 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:36.857329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:36.857392 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:36.857782 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:36.857783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:36.857828 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:10:36.857920 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:10:36.858703 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":31,"thread_start_us":49,"threads_started":1}
W20260811 07:10:36.859468 333 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:36.859524 333 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce; no voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, bd3c3dd6268246a2bc14b4078f4ad52c
I20260811 07:10:36.859639 986 raft_consensus.cc:2726] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:10:37.169447 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:37.169292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:57091)
0811 07:10:37.169363 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:37.169405 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:10:37.252188 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:37.251951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:38001)
0811 07:10:37.252089 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:37.252140 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260811 07:10:37.398578 998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:37.398359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:58437)
0811 07:10:37.398492 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:37.398536 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260811 07:10:38.170166 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:38.169950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:43697)
0811 07:10:38.170081 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:38.170117 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260811 07:10:38.252871 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:38.252647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:58595)
0811 07:10:38.252788 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:38.252830 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260811 07:10:38.399428 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:38.399133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:40133)
0811 07:10:38.399330 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:38.399377 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":100,"threads_started":1}
I20260811 07:10:38.872160 1003 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:38.872313 1003 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:38.872530 1003 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
W20260811 07:10:38.873030 332 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
W20260811 07:10:38.873101 333 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:38.873131 332 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce; no voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, bd3c3dd6268246a2bc14b4078f4ad52c
I20260811 07:10:38.873219 1003 raft_consensus.cc:2726] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:10:39.170836 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:39.170627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:57959)
0811 07:10:39.170750 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:39.170786 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260811 07:10:39.253597 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:39.253328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:45639)
0811 07:10:39.253501 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:39.253535 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260811 07:10:39.400390 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:39.400016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:52707)
0811 07:10:39.400303 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:39.400343 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":203,"threads_started":1}
I20260811 07:10:40.171627 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:40.171386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:39839)
0811 07:10:40.171534 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:40.171576 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260811 07:10:40.254354 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:40.254141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:33849)
0811 07:10:40.254274 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:40.254306 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260811 07:10:40.401180 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:40.400933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:60173)
0811 07:10:40.401098 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:40.401135 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260811 07:10:41.172369 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:41.172107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:54133)
0811 07:10:41.172241 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:41.172320 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260811 07:10:41.255003 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:41.254759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:33403)
0811 07:10:41.254915 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:41.254955 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260811 07:10:41.401891 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:41.401647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:57275)
0811 07:10:41.401797 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:41.401844 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260811 07:10:41.471022 1022 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:41.471181 1022 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:41.471441 1022 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
W20260811 07:10:41.471858 332 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
W20260811 07:10:41.471952 333 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:41.472012 333 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce; no voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, bd3c3dd6268246a2bc14b4078f4ad52c
I20260811 07:10:41.472096 1022 raft_consensus.cc:2726] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:10:42.173050 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:42.172824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:40519)
0811 07:10:42.172957 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:42.173000 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260811 07:10:42.255748 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:42.255507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:41337)
0811 07:10:42.255662 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:42.255702 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260811 07:10:42.402582 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:42.402350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:52701)
0811 07:10:42.402498 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:42.402537 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260811 07:10:43.173895 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:43.173654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:42239)
0811 07:10:43.173800 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:43.173844 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260811 07:10:43.256532 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:43.256212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:38107)
0811 07:10:43.256446 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:43.256486 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260811 07:10:43.403391 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:43.403107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:46353)
0811 07:10:43.403269 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:43.403315 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260811 07:10:44.174654 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:44.174423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:34143)
0811 07:10:44.174562 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:44.174603 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260811 07:10:44.221230 1033 raft_consensus.cc:482] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:44.221324 1033 raft_consensus.cc:504] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c09ecebfc8b4e28ae8be7e69b7e9fdd" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44809 } } peers { permanent_uuid: "bd3c3dd6268246a2bc14b4078f4ad52c" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 38059 } } peers { permanent_uuid: "8e608cb5a7d143c08591c538250f02ce" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 45957 } }
I20260811 07:10:44.221544 1033 leader_election.cc:288] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809), bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059)
W20260811 07:10:44.222019 333 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd3c3dd6268246a2bc14b4078f4ad52c (127.3.236.131:38059): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
W20260811 07:10:44.222084 332 leader_election.cc:341] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c09ecebfc8b4e28ae8be7e69b7e9fdd (127.3.236.130:44809): Not found: Tablet not found: 37575fb8352b40b89718ab8faaf1cd7b
I20260811 07:10:44.222119 332 leader_election.cc:302] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e608cb5a7d143c08591c538250f02ce; no voters: 3c09ecebfc8b4e28ae8be7e69b7e9fdd, bd3c3dd6268246a2bc14b4078f4ad52c
I20260811 07:10:44.222214 1033 raft_consensus.cc:2726] T 37575fb8352b40b89718ab8faaf1cd7b P 8e608cb5a7d143c08591c538250f02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:10:44.257256 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:44.257001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:59223)
0811 07:10:44.257173 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:44.257210 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
I20260811 07:10:44.404157 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:44.403892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:49495)
0811 07:10:44.404063 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:44.404103 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260811 07:10:45.175361 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:45.175152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:52129)
0811 07:10:45.175279 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:45.175312 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260811 07:10:45.258118 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:45.257888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:43979)
0811 07:10:45.258031 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:45.258069 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260811 07:10:45.404994 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:45.404700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:45497)
0811 07:10:45.404882 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:45.404935 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260811 07:10:46.176420 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:46.175955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:42457)
0811 07:10:46.176149 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:46.176197 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260811 07:10:46.258888 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:46.258652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:43449)
0811 07:10:46.258793 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:46.258836 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260811 07:10:46.405719 1045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:46.405478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:58873)
0811 07:10:46.405633 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:46.405671 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260811 07:10:47.177124 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:47.176904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:52065)
0811 07:10:47.177035 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:47.177074 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260811 07:10:47.259570 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:47.259359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:55395)
0811 07:10:47.259484 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:47.259522 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260811 07:10:47.406411 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:47.406168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:41809)
0811 07:10:47.406326 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:47.406364 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
I20260811 07:10:48.177932 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:48.177648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:47989)
0811 07:10:48.177801 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:48.177848 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260811 07:10:48.260239 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:48.260013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:58297)
0811 07:10:48.260151 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:48.260192 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260811 07:10:48.407133 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:48.406896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:58661)
0811 07:10:48.407045 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:48.407085 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260811 07:10:49.178829 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:49.178591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:58381)
0811 07:10:49.178729 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:49.178778 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260811 07:10:49.260972 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:49.260763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:50675)
0811 07:10:49.260884 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:49.260923 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260811 07:10:49.407903 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:49.407654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:55837)
0811 07:10:49.407811 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:49.407855 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260811 07:10:50.179687 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:50.179448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.129:39397)
0811 07:10:50.179590 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:50.179635 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260811 07:10:50.261777 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:50.261536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.130:34865)
0811 07:10:50.261682 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:50.261724 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260811 07:10:50.404124 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 32709
I20260811 07:10:50.414867 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:50.413322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35667 (local address 127.3.236.131:59449)
0811 07:10:50.414780 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:50.414822 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.190:35667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1438,"thread_start_us":61,"threads_started":1}
I20260811 07:10:50.421113 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 518
I20260811 07:10:50.429086 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 751
2026-08-11T07:10:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58970 ms)
[----------] 30 tests from SecurityITest (243899 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:10:50 dist-test-slave-2d19 krb5kdc[1070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:10:50 dist-test-slave-2d19 krb5kdc[1070](info): set up 2 sockets
Aug 11 07:10:50 dist-test-slave-2d19 krb5kdc[1070](info): commencing operation
krb5kdc: starting...
W20260811 07:10:51.728184 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:10:51 dist-test-slave-2d19 krb5kdc[1070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:10:51.854866 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43523
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:43523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:51.938580 1092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:51.938836 1092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:51.938874 1092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:51.940032 1092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:51.940101 1092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:51.940124 1092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:51.940145 1092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:51.940177 1092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:51.941708 1092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:43523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:51.941999 1092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:51.942232 1092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:51.947852 1103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:51.947973 1101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:51.947852 1106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:51.948002 1092 server_base.cc:1034] running on GCE node
I20260811 07:10:51.948457 1092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:51.948766 1092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:51.949921 1092 hybrid_clock.cc:648] HybridClock initialized: now 1786432251949899 us; error 41 us; skew 500 ppm
Aug 11 07:10:51 dist-test-slave-2d19 krb5kdc[1070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432251, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:51.952745 1092 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:10:51.953251 1092 webserver.cc:460] Webserver started at http://127.3.236.190:43275/ using document root <none> and password file <none>
I20260811 07:10:51.953512 1092 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:51.953583 1092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:51.953729 1092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:51.954885 1092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "43ac21a9360146a4b1372be3c16ed168"
format_stamp: "Formatted at 2026-08-11 07:10:51 on dist-test-slave-2d19"
server_key: "838e26e7da13fd2e80146edbb95accc1"
server_key_iv: "7eb69a0be99681444f3cda52c8f96a4a"
server_key_version: "encryptionkey@0"
I20260811 07:10:51.955264 1092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "43ac21a9360146a4b1372be3c16ed168"
format_stamp: "Formatted at 2026-08-11 07:10:51 on dist-test-slave-2d19"
server_key: "838e26e7da13fd2e80146edbb95accc1"
server_key_iv: "7eb69a0be99681444f3cda52c8f96a4a"
server_key_version: "encryptionkey@0"
I20260811 07:10:51.956753 1092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:10:51.957510 1117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:51.957736 1092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:51.957856 1092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "43ac21a9360146a4b1372be3c16ed168"
format_stamp: "Formatted at 2026-08-11 07:10:51 on dist-test-slave-2d19"
server_key: "838e26e7da13fd2e80146edbb95accc1"
server_key_iv: "7eb69a0be99681444f3cda52c8f96a4a"
server_key_version: "encryptionkey@0"
I20260811 07:10:51.957988 1092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:51.970194 1092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:51.972155 1092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:51.972450 1092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:51.977164 1092 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:43523
I20260811 07:10:51.977201 1223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:43523 every 8 connection(s)
I20260811 07:10:51.977608 1092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:51.978605 1224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:51.980094 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 1092
I20260811 07:10:51.980233 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:51.980489 4018 external_mini_cluster.cc:1428] Setting key a9a40ccdf039d704aa3e44f19370e6eb
I20260811 07:10:51.981328 1224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: Bootstrap starting.
I20260811 07:10:51.982062 1224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:51.982401 1224 log.cc:824] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:51.983422 1224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: No bootstrap required, opened a new log
I20260811 07:10:51.985766 1224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } }
I20260811 07:10:51.985963 1224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:51.986047 1224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ac21a9360146a4b1372be3c16ed168, State: Initialized, Role: FOLLOWER
I20260811 07:10:51.986214 1224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } }
I20260811 07:10:51.986326 1224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:51.986364 1224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:51.986413 1224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:51.987020 1224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } }
I20260811 07:10:51.987113 1224 leader_election.cc:302] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43ac21a9360146a4b1372be3c16ed168; no voters:
I20260811 07:10:51.987303 1224 leader_election.cc:288] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:51.987337 1236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:51.987454 1236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [term 1 LEADER]: Becoming Leader. State: Replica: 43ac21a9360146a4b1372be3c16ed168, State: Running, Role: LEADER
I20260811 07:10:51.987586 1224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:51.987564 1236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } }
I20260811 07:10:51.988087 1240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43ac21a9360146a4b1372be3c16ed168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } } }
I20260811 07:10:51.988085 1236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ac21a9360146a4b1372be3c16ed168. Latest consensus state: current_term: 1 leader_uuid: "43ac21a9360146a4b1372be3c16ed168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ac21a9360146a4b1372be3c16ed168" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 43523 } } }
I20260811 07:10:51.988333 1236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [sys.catalog]: This master's current role is: LEADER
Aug 11 07:10:51 dist-test-slave-2d19 krb5kdc[1070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:51.988574 1240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168 [sys.catalog]: This master's current role is: LEADER
W20260811 07:10:51.989842 1261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:10:51.989924 1261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:10:51.990036 1249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:51.990232 1249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:51.991727 1249 catalog_manager.cc:1347] Generated new cluster ID: f66bf60e79574c3693f003b8c6782f4d
I20260811 07:10:51.991781 1249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:51.993605 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:51.981037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.3.236.190:43523)
0811 07:10:51.981249 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:51.981259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:51.981291 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:10:51.981894 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:51.981903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:51.981980 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:51.982148 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:51.982871 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:51.984436 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:51.985206 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:51.985875 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:51.986004 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:51.989008 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:51.989012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:51.989012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:51.989025 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:51.991797 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:51.992243 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:51.992246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:51.992246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:51.992372 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:51.992631 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:51.992631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:51.992632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:51.992963 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:51.993101 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:51.993247 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 07:10:51.993383 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260811 07:10:52.005306 1249 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:52.006042 1249 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:52.011833 1249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: Generated new TSK 0
I20260811 07:10:52.012027 1249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:10:52.044432 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:52.158061 1269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:52.158306 1269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:52.158345 1269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:52.159509 1269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:52.159575 1269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:52.159621 1269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:52.161389 1269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:52.161688 1269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:52.161983 1269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:52.167227 1274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:52.167232 1276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:52.167275 1278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:52.167249 1269 server_base.cc:1034] running on GCE node
I20260811 07:10:52.167711 1269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:52.167972 1269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:52.169170 1269 hybrid_clock.cc:648] HybridClock initialized: now 1786432252169158 us; error 33 us; skew 500 ppm
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:52.172113 1269 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:10:52.172676 1269 webserver.cc:460] Webserver started at http://127.3.236.129:37891/ using document root <none> and password file <none>
I20260811 07:10:52.172942 1269 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:52.173012 1269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:52.173157 1269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:52.174306 1269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "85312b47cf1b453e96c77a7662190a46"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "99bb64bf53d2e77c4eb6ab204a2c9b1d"
server_key_iv: "db492757fe913789e84e55171740bea3"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.174690 1269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "85312b47cf1b453e96c77a7662190a46"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "99bb64bf53d2e77c4eb6ab204a2c9b1d"
server_key_iv: "db492757fe913789e84e55171740bea3"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.176215 1269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:52.177093 1290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.177256 1269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:52.177357 1269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "85312b47cf1b453e96c77a7662190a46"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "99bb64bf53d2e77c4eb6ab204a2c9b1d"
server_key_iv: "db492757fe913789e84e55171740bea3"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.177440 1269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:52.195101 1269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:52.197525 1269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:52.197691 1269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:52.197948 1269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:52.198263 1269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:52.198316 1269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.198359 1269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:52.198383 1269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.204730 1269 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:37975
I20260811 07:10:52.204782 1479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:37975 every 8 connection(s)
I20260811 07:10:52.205101 1269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:52.211408 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 1269
I20260811 07:10:52.211522 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:52.211725 4018 external_mini_cluster.cc:1428] Setting key b3914e9579f8cd56649c810a6006b137
I20260811 07:10:52.214664 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.205829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:39713 (local address 127.3.236.190:43523)
0811 07:10:52.205877 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.205879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.206088 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:10:52.206626 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.206627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.206639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.206705 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:52.207670 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.208209 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.209256 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.209454 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.209508 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.211712 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:52.211713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:52.211714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:52.211722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:52.213267 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.213670 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.213671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.213671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.213728 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.213991 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.213992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.213993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.214100 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:52.214168 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.214443 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.214564 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:10:52.214882 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.205782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43523 (local address 127.3.236.129:39713)
0811 07:10:52.205947 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.205991 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.206507 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.206782 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.206786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.207354 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:52.207571 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.207575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.208339 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.208340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.209179 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.209181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.209528 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.209529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.209636 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.210167 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:52.210175 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:52.211621 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:52.213352 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.213359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.213362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.213593 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.213794 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.213795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.213795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.213910 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.214173 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:52.214181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:52.214314 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.214421 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.214631 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":102,"threads_started":1}
I20260811 07:10:52.215370 1481 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43523
I20260811 07:10:52.215494 1481 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:52.215704 1481 heartbeater.cc:511] Master 127.3.236.190:43523 requested a full tablet report, sending...
I20260811 07:10:52.216245 1150 ts_manager.cc:194] Registered new tserver with Master: 85312b47cf1b453e96c77a7662190a46 (127.3.236.129:37975)
I20260811 07:10:52.216895 1150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:39713
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:10:52.239696 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:52.325994 1490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:52.326205 1490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:52.326228 1490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:52.327310 1490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:52.327347 1490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:52.327374 1490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:52.328804 1490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:52.328994 1490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:52.329176 1490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:10:52.334403 1490 server_base.cc:1034] running on GCE node
W20260811 07:10:52.334348 1498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:52.334379 1499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:52.334363 1502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:52.334820 1490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:52.335139 1490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:52.336306 1490 hybrid_clock.cc:648] HybridClock initialized: now 1786432252336289 us; error 35 us; skew 500 ppm
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:52.338958 1490 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:10:52.339370 1490 webserver.cc:460] Webserver started at http://127.3.236.130:37999/ using document root <none> and password file <none>
I20260811 07:10:52.339635 1490 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:52.339685 1490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:52.339818 1490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:52.340951 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "b6c48a78d1197e2ea4ba0693d1a434f1"
server_key_iv: "b160a4ddefa86d35169f2a029ae1efee"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.341341 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "b6c48a78d1197e2ea4ba0693d1a434f1"
server_key_iv: "b160a4ddefa86d35169f2a029ae1efee"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.342757 1490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:10:52.343565 1510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.343736 1490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:52.343843 1490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "b6c48a78d1197e2ea4ba0693d1a434f1"
server_key_iv: "b160a4ddefa86d35169f2a029ae1efee"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.343930 1490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:52.360927 1490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:52.363104 1490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:52.363276 1490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:52.363586 1490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:52.363938 1490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:52.363983 1490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.364034 1490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:52.364060 1490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.370332 1490 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:44509
I20260811 07:10:52.370391 1684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:44509 every 8 connection(s)
I20260811 07:10:52.370770 1490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:52.375589 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 1490
I20260811 07:10:52.375723 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:10:52.375967 4018 external_mini_cluster.cc:1428] Setting key 9ceea052fb3354048e902cb9fb8e1edb
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:52.381306 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.371565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:55561 (local address 127.3.236.190:43523)
0811 07:10:52.371631 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.371634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.371920 (+ 286us) server_negotiation.cc:407] Connection header received
0811 07:10:52.372483 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.372484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.372494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.372551 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:52.373544 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.374053 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.375029 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.375214 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.375267 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.378158 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:52.378159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:52.378159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:52.378167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:52.379613 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.380183 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.380185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.380185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.380272 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.380533 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.380534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.380535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.380687 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:52.380766 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.381013 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.381111 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:10:52.381341 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.371553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43523 (local address 127.3.236.130:55561)
0811 07:10:52.371747 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.371804 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.372385 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.372636 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.372640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.373246 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:52.373451 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.373456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.374136 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.374136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.374954 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.374956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.375313 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.375314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.375447 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.376115 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:52.376145 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:52.378024 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:52.379707 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.379715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.379718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.380032 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.380347 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.380347 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.380348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.380451 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.380758 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:52.380763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:52.380884 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.380969 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.381101 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260811 07:10:52.382120 1686 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43523
I20260811 07:10:52.382246 1686 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:52.382472 1686 heartbeater.cc:511] Master 127.3.236.190:43523 requested a full tablet report, sending...
I20260811 07:10:52.382912 1150 ts_manager.cc:194] Registered new tserver with Master: 51d48cae2c9741fbbbc9ab87c22b6a5a (127.3.236.130:44509)
I20260811 07:10:52.383315 1150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:55561
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:10:52.404896 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:52.492296 1695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:52.492534 1695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:52.492575 1695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:52.493736 1695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:52.493798 1695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:52.493842 1695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:52.495343 1695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:52.495611 1695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:52.495837 1695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:52.501168 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:52.501175 1704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:52.501223 1695 server_base.cc:1034] running on GCE node
W20260811 07:10:52.501175 1706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:52.501622 1695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:52.501834 1695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:52.503008 1695 hybrid_clock.cc:648] HybridClock initialized: now 1786432252502987 us; error 31 us; skew 500 ppm
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:52.505635 1695 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:10:52.506028 1695 webserver.cc:460] Webserver started at http://127.3.236.131:41741/ using document root <none> and password file <none>
I20260811 07:10:52.506263 1695 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:52.506312 1695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:52.506438 1695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:52.507485 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "482925dd911148e5911f9c80248c4ed0"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "7da72027a2d098fd2b4db3b37f951cfd"
server_key_iv: "1920bf5f455a77c9101c87349620054f"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.507921 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "482925dd911148e5911f9c80248c4ed0"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "7da72027a2d098fd2b4db3b37f951cfd"
server_key_iv: "1920bf5f455a77c9101c87349620054f"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.509359 1695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:52.510138 1715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.510319 1695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:52.510439 1695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "482925dd911148e5911f9c80248c4ed0"
format_stamp: "Formatted at 2026-08-11 07:10:52 on dist-test-slave-2d19"
server_key: "7da72027a2d098fd2b4db3b37f951cfd"
server_key_iv: "1920bf5f455a77c9101c87349620054f"
server_key_version: "encryptionkey@0"
I20260811 07:10:52.510514 1695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:52.529100 1695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:52.531293 1695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:52.531467 1695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:52.531729 1695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:52.532085 1695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:52.532147 1695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.532191 1695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:52.532222 1695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:52.538409 1695 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:40055
I20260811 07:10:52.538480 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:40055 every 8 connection(s)
I20260811 07:10:52.538774 1695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:10:52.540956 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 1695
I20260811 07:10:52.541065 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:10:52.541232 4018 external_mini_cluster.cc:1428] Setting key 578d0a0d88fab2d70167999955bf36d7
Aug 11 07:10:52 dist-test-slave-2d19 krb5kdc[1070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432252, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:52.548413 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.539539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:33585 (local address 127.3.236.190:43523)
0811 07:10:52.539605 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.539608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.539806 (+ 198us) server_negotiation.cc:407] Connection header received
0811 07:10:52.540355 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.540356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.540366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.540431 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:52.541502 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.542266 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.543238 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.543427 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.543487 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.545585 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:52.545586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:52.545587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:52.545593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:52.547063 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.547520 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.547521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.547521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.547577 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.547782 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.547783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.547783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.547882 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:52.547931 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.548188 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.548298 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:10:52.548552 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.539512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:43523 (local address 127.3.236.131:33585)
0811 07:10:52.539676 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.539718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.540203 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.540491 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.540494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.541142 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:52.541395 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.541401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.542354 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.542355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.543160 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.543162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.543506 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.543507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.543606 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.544113 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:52.544122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:52.545502 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:52.547147 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.547153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.547155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.547446 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.547634 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:52.547635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:52.547635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:52.547731 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:52.547933 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:52.547937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:52.548052 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.548134 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.548297 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260811 07:10:52.549042 1891 heartbeater.cc:347] Connected to a master server at 127.3.236.190:43523
I20260811 07:10:52.549153 1891 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:52.549337 1891 heartbeater.cc:511] Master 127.3.236.190:43523 requested a full tablet report, sending...
I20260811 07:10:52.549703 1150 ts_manager.cc:194] Registered new tserver with Master: 482925dd911148e5911f9c80248c4ed0 (127.3.236.131:40055)
I20260811 07:10:52.550072 1150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:33585
I20260811 07:10:52.552958 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:52.559803 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.554334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.3.236.190:43523)
0811 07:10:52.554387 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.554389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.554505 (+ 116us) server_negotiation.cc:407] Connection header received
0811 07:10:52.554546 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.554546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.554554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.554605 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:52.555098 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.555555 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.556312 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.556508 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.556564 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.557609 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:52.557610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:52.557611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:52.557616 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:52.558922 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.559205 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.559206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.559206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.559255 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:52.559401 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:52.559401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:52.559401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:52.559536 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:52.559606 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.559685 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.559744 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 07:10:52.561350 1150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:52.563112 1150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:52.569623 1909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44509 (local address 127.0.0.1:50298)
0811 07:10:52.565623 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.565668 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.565732 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.565958 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.565962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.565964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:52.566031 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.566032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567518 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.567519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.568616 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.568618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569293 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.569378 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569397 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.569482 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.569546 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":51,"threads_started":1}
I20260811 07:10:52.569728 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.3.236.131:40055)
0811 07:10:52.565984 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.565989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.565998 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:52.566035 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.566037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.566051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.566122 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:52.566317 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567097 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.568642 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569370 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569449 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569519 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.569589 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.569629 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":77,"threads_started":1}
I20260811 07:10:52.569840 1908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:37975 (local address 127.0.0.1:57398)
0811 07:10:52.566143 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.566153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.566204 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.566472 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.566474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.566476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:52.566522 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.566523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567729 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.567730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.568659 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.568661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569586 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.569655 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569660 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.569711 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.569758 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":77,"threads_started":1}
I20260811 07:10:52.569897 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.3.236.130:44509)
0811 07:10:52.565762 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.565768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.565780 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:10:52.565831 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.565833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.565850 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.565939 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:52.566102 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567405 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.568698 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569194 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569280 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569341 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.569756 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.569807 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
I20260811 07:10:52.570070 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.3.236.129:37975)
0811 07:10:52.566097 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:52.566103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:52.566262 (+ 159us) server_negotiation.cc:407] Connection header received
0811 07:10:52.566309 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:52.566311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:52.566326 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:52.566399 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:52.566604 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567620 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.568752 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569443 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569503 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569550 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:52.569926 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 07:10:52.569988 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
I20260811 07:10:52.570377 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:52.565450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:40055 (local address 127.0.0.1:59010)
0811 07:10:52.565799 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:52.565810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:52.565862 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:52.566200 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:52.566201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:52.566204 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:52.566250 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.566252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.567212 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.567213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.568553 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:52.568556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:52.569459 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:52.569460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:52.569519 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:52.569522 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:52.569574 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:10:52.569603 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":54,"thread_start_us":120,"threads_started":1}
I20260811 07:10:52.570606 1592 tablet_service.cc:1467] Processing CreateTablet for tablet 49b5b51af3df4336b2cab9c771360369 (DEFAULT_TABLE table=test-table [id=5ff032b7cf9e4a2d8bf99a41979489cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:52.570999 1592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b5b51af3df4336b2cab9c771360369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:52.570964 1374 tablet_service.cc:1467] Processing CreateTablet for tablet 49b5b51af3df4336b2cab9c771360369 (DEFAULT_TABLE table=test-table [id=5ff032b7cf9e4a2d8bf99a41979489cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:52.571302 1374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b5b51af3df4336b2cab9c771360369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:52.571202 1790 tablet_service.cc:1467] Processing CreateTablet for tablet 49b5b51af3df4336b2cab9c771360369 (DEFAULT_TABLE table=test-table [id=5ff032b7cf9e4a2d8bf99a41979489cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:52.571516 1790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b5b51af3df4336b2cab9c771360369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:52.573798 1914 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Bootstrap starting.
I20260811 07:10:52.574451 1914 tablet_bootstrap.cc:654] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:52.574733 1920 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Bootstrap starting.
I20260811 07:10:52.575004 1915 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Bootstrap starting.
I20260811 07:10:52.575115 1914 log.cc:824] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:52.575663 1915 tablet_bootstrap.cc:654] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:52.575955 1914 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: No bootstrap required, opened a new log
I20260811 07:10:52.576040 1914 ts_tablet_manager.cc:1397] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:52.576325 1915 log.cc:824] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:52.576944 1920 tablet_bootstrap.cc:654] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:52.577231 1915 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: No bootstrap required, opened a new log
I20260811 07:10:52.577306 1915 ts_tablet_manager.cc:1397] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:10:52.577544 1920 log.cc:824] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:52.578280 1920 tablet_bootstrap.cc:492] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: No bootstrap required, opened a new log
I20260811 07:10:52.578357 1920 ts_tablet_manager.cc:1397] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:10:52.578639 1914 raft_consensus.cc:348] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.578830 1914 raft_consensus.cc:374] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:52.578877 1914 raft_consensus.cc:729] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d48cae2c9741fbbbc9ab87c22b6a5a, State: Initialized, Role: FOLLOWER
I20260811 07:10:52.579015 1914 consensus_queue.cc:260] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.579365 1686 heartbeater.cc:503] Master 127.3.236.190:43523 was elected leader, sending a full tablet report...
I20260811 07:10:52.579370 1914 ts_tablet_manager.cc:1428] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:52.579674 1915 raft_consensus.cc:348] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.579804 1915 raft_consensus.cc:374] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:52.579847 1915 raft_consensus.cc:729] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482925dd911148e5911f9c80248c4ed0, State: Initialized, Role: FOLLOWER
I20260811 07:10:52.579967 1915 consensus_queue.cc:260] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.580245 1891 heartbeater.cc:503] Master 127.3.236.190:43523 was elected leader, sending a full tablet report...
I20260811 07:10:52.580243 1915 ts_tablet_manager.cc:1428] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 07:10:52.580330 1920 raft_consensus.cc:348] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.580546 1920 raft_consensus.cc:374] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:52.580595 1920 raft_consensus.cc:729] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85312b47cf1b453e96c77a7662190a46, State: Initialized, Role: FOLLOWER
I20260811 07:10:52.580745 1920 consensus_queue.cc:260] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:52.581039 1920 ts_tablet_manager.cc:1428] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:52.581168 1481 heartbeater.cc:503] Master 127.3.236.190:43523 was elected leader, sending a full tablet report...
W20260811 07:10:52.621471 1687 tablet.cc:2369] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:10:52.705868 1484 tablet.cc:2369] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:10:52.789386 1892 tablet.cc:2369] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:10:53.015974 1932 raft_consensus.cc:482] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:53.016110 1932 raft_consensus.cc:504] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:53.016585 1932 leader_election.cc:288] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51d48cae2c9741fbbbc9ab87c22b6a5a (127.3.236.130:44509), 482925dd911148e5911f9c80248c4ed0 (127.3.236.131:40055)
I20260811 07:10:53.019572 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.016846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:52359 (local address 127.3.236.130:44509)
0811 07:10:53.016921 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.016924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.017027 (+ 103us) server_negotiation.cc:407] Connection header received
0811 07:10:53.017102 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.017104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.017122 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.017194 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.017360 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.017846 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.018870 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.019226 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.019306 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.019316 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.019466 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.019501 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:10:53.019589 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.016749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:44509 (local address 127.3.236.129:52359)
0811 07:10:53.016953 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.016964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.017045 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.017201 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.017202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.017203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.017285 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.017291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.017937 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.017938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.018781 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.018783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.019318 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.019319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.019391 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.019400 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.019461 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.019501 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":38,"thread_start_us":92,"threads_started":1}
I20260811 07:10:53.019654 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.016918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:43721 (local address 127.3.236.131:40055)
0811 07:10:53.016971 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.016974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.017045 (+ 71us) server_negotiation.cc:407] Connection header received
0811 07:10:53.017128 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.017129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.017140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.017207 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.017381 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.017848 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.018861 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.019272 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.019357 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.019366 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.019542 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.019582 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:10:53.019709 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.016802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:40055 (local address 127.3.236.129:43721)
0811 07:10:53.016984 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.016995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.017063 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.017211 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.017212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.017213 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.017304 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.017306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.017941 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.017942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.018770 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.018772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.019370 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.019372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.019454 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.019458 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.019514 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.019558 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":21,"thread_start_us":66,"threads_started":1}
I20260811 07:10:53.020105 1821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b5b51af3df4336b2cab9c771360369" candidate_uuid: "85312b47cf1b453e96c77a7662190a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482925dd911148e5911f9c80248c4ed0" is_pre_election: true
I20260811 07:10:53.020318 1821 raft_consensus.cc:2454] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85312b47cf1b453e96c77a7662190a46 in term 0.
I20260811 07:10:53.020430 1617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b5b51af3df4336b2cab9c771360369" candidate_uuid: "85312b47cf1b453e96c77a7662190a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" is_pre_election: true
I20260811 07:10:53.020601 1617 raft_consensus.cc:2454] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85312b47cf1b453e96c77a7662190a46 in term 0.
I20260811 07:10:53.020594 1293 leader_election.cc:302] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482925dd911148e5911f9c80248c4ed0, 85312b47cf1b453e96c77a7662190a46; no voters:
I20260811 07:10:53.020716 1932 raft_consensus.cc:2781] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:53.020805 1932 raft_consensus.cc:482] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:53.020849 1932 raft_consensus.cc:3037] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.021605 1932 raft_consensus.cc:504] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:53.021754 1932 leader_election.cc:288] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [CANDIDATE]: Term 1 election: Requested vote from peers 51d48cae2c9741fbbbc9ab87c22b6a5a (127.3.236.130:44509), 482925dd911148e5911f9c80248c4ed0 (127.3.236.131:40055)
I20260811 07:10:53.021927 1821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b5b51af3df4336b2cab9c771360369" candidate_uuid: "85312b47cf1b453e96c77a7662190a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482925dd911148e5911f9c80248c4ed0"
I20260811 07:10:53.021940 1617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b5b51af3df4336b2cab9c771360369" candidate_uuid: "85312b47cf1b453e96c77a7662190a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a"
I20260811 07:10:53.022028 1821 raft_consensus.cc:3037] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.022042 1617 raft_consensus.cc:3037] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.022946 1821 raft_consensus.cc:2454] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85312b47cf1b453e96c77a7662190a46 in term 1.
I20260811 07:10:53.022946 1617 raft_consensus.cc:2454] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85312b47cf1b453e96c77a7662190a46 in term 1.
I20260811 07:10:53.023126 1293 leader_election.cc:302] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482925dd911148e5911f9c80248c4ed0, 85312b47cf1b453e96c77a7662190a46; no voters:
I20260811 07:10:53.023236 1932 raft_consensus.cc:2781] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:53.023412 1932 raft_consensus.cc:686] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [term 1 LEADER]: Becoming Leader. State: Replica: 85312b47cf1b453e96c77a7662190a46, State: Running, Role: LEADER
I20260811 07:10:53.023535 1932 consensus_queue.cc:237] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } }
I20260811 07:10:53.024447 1150 catalog_manager.cc:5495] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85312b47cf1b453e96c77a7662190a46 (127.3.236.129). New cstate: current_term: 1 leader_uuid: "85312b47cf1b453e96c77a7662190a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85312b47cf1b453e96c77a7662190a46" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 } health_report { overall_health: UNKNOWN } } }
I20260811 07:10:53.119052 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.116295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.3.236.129:37975)
0811 07:10:53.116420 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.116422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.116434 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:10:53.116501 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.116501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.116523 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.116592 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:53.116755 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.117284 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.118151 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.118363 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.118427 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.118577 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:53.118690 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:53.118767 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.118858 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.118935 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260811 07:10:53.121043 1617 raft_consensus.cc:1261] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 1 FOLLOWER]: Refusing update from remote peer 85312b47cf1b453e96c77a7662190a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:53.121071 1821 raft_consensus.cc:1261] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 1 FOLLOWER]: Refusing update from remote peer 85312b47cf1b453e96c77a7662190a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:53.121469 1935 consensus_queue.cc:1035] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51d48cae2c9741fbbbc9ab87c22b6a5a" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:53.121621 1932 consensus_queue.cc:1035] T 49b5b51af3df4336b2cab9c771360369 P 85312b47cf1b453e96c77a7662190a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482925dd911148e5911f9c80248c4ed0" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:53.124285 1943 mvcc.cc:205] Tried to move back new op lower bound from 7317226508780376064 to 7317226508384874496. Current Snapshot: MvccSnapshot[applied={T|T < 7317226508780376064}]
I20260811 07:10:53.124447 1949 mvcc.cc:205] Tried to move back new op lower bound from 7317226508780376064 to 7317226508384874496. Current Snapshot: MvccSnapshot[applied={T|T < 7317226508780376064}]
I20260811 07:10:53.127271 1150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:53.127374 1150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:53.128216 1150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43ac21a9360146a4b1372be3c16ed168: Sending DeleteTablet for 3 replicas of tablet 49b5b51af3df4336b2cab9c771360369
I20260811 07:10:53.128633 1790 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b5b51af3df4336b2cab9c771360369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:59010
I20260811 07:10:53.128698 1592 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b5b51af3df4336b2cab9c771360369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:50298
I20260811 07:10:53.128926 1374 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b5b51af3df4336b2cab9c771360369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:57398
I20260811 07:10:53.129079 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 1269
I20260811 07:10:53.146003 1976 tablet_replica.cc:330] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: stopping tablet replica
I20260811 07:10:53.146199 1976 raft_consensus.cc:2229] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:53.146337 1976 raft_consensus.cc:2258] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:53.147219 1976 ts_tablet_manager.cc:1905] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:53.148835 1976 ts_tablet_manager.cc:1918] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:53.148908 1976 log.cc:1196] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/49b5b51af3df4336b2cab9c771360369
I20260811 07:10:53.149143 1976 ts_tablet_manager.cc:1939] T 49b5b51af3df4336b2cab9c771360369 P 51d48cae2c9741fbbbc9ab87c22b6a5a: Deleting consensus metadata
I20260811 07:10:53.151476 1974 tablet_replica.cc:330] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: stopping tablet replica
I20260811 07:10:53.151487 1123 catalog_manager.cc:4841] TS 51d48cae2c9741fbbbc9ab87c22b6a5a (127.3.236.130:44509): tablet 49b5b51af3df4336b2cab9c771360369 (table test-table [id=5ff032b7cf9e4a2d8bf99a41979489cc]) successfully deleted
I20260811 07:10:53.151645 1974 raft_consensus.cc:2229] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:53.151783 1974 raft_consensus.cc:2258] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:53.152209 1974 ts_tablet_manager.cc:1905] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:53.153738 1974 ts_tablet_manager.cc:1918] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:53.153836 1974 log.cc:1196] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/49b5b51af3df4336b2cab9c771360369
I20260811 07:10:53.154102 1974 ts_tablet_manager.cc:1939] T 49b5b51af3df4336b2cab9c771360369 P 482925dd911148e5911f9c80248c4ed0: Deleting consensus metadata
I20260811 07:10:53.154553 1119 catalog_manager.cc:4841] TS 482925dd911148e5911f9c80248c4ed0 (127.3.236.131:40055): tablet 49b5b51af3df4336b2cab9c771360369 (table test-table [id=5ff032b7cf9e4a2d8bf99a41979489cc]) successfully deleted
W20260811 07:10:53.155869 1123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:37975 (error 108)
I20260811 07:10:53.156095 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 1490
I20260811 07:10:53.158828 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.158639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:37975 (local address 127.0.0.1:57404)
0811 07:10:53.158743 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.158793 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":58,"threads_started":1}
W20260811 07:10:53.159001 1123 catalog_manager.cc:4584] TS 85312b47cf1b453e96c77a7662190a46 (127.3.236.129:37975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49b5b51af3df4336b2cab9c771360369: Network error: Client connection negotiation failed: client connection to 127.3.236.129:37975: connect: Connection refused (error 111)
I20260811 07:10:53.169167 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 1695
I20260811 07:10:53.183637 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 1092
2026-08-11T07:10:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2753 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T07:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:53Z Disabled control of system clock
I20260811 07:10:53.223654 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36717
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:37341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:36717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:53.310451 1985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:53.310698 1985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:53.310719 1985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:53.311913 1985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:53.311968 1985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:53.311981 1985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:53.311995 1985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:53.312006 1985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:53.313403 1985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:36717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:53.313683 1985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:53.314002 1985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:53.318919 2000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.318984 1995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.318919 1994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.319263 1985 server_base.cc:1034] running on GCE node
I20260811 07:10:53.319555 1985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:53.319870 1985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:53.321031 1985 hybrid_clock.cc:648] HybridClock initialized: now 1786432253321020 us; error 32 us; skew 500 ppm
I20260811 07:10:53.322392 1985 webserver.cc:460] Webserver started at http://127.3.236.190:46323/ using document root <none> and password file <none>
I20260811 07:10:53.322680 1985 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:53.322757 1985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:53.322902 1985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:53.323997 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "c0a0040dd5584f2d86bf474a2d817cdc"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "c48b5ce17aec1978809feb9d93d92135"
server_key_iv: "1b8f2dd25b2f338b3fcdbd9791126146"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.324422 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "c0a0040dd5584f2d86bf474a2d817cdc"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "c48b5ce17aec1978809feb9d93d92135"
server_key_iv: "1b8f2dd25b2f338b3fcdbd9791126146"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.325860 1985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:53.326666 2010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.326848 1985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:53.326949 1985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "c0a0040dd5584f2d86bf474a2d817cdc"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "c48b5ce17aec1978809feb9d93d92135"
server_key_iv: "1b8f2dd25b2f338b3fcdbd9791126146"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.327028 1985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:53.343022 1985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:53.343397 1985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:53.343569 1985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:53.348037 1985 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:36717
I20260811 07:10:53.348166 2115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:36717 every 8 connection(s)
I20260811 07:10:53.348546 1985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:53.349956 2116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:53.352427 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: Bootstrap starting.
I20260811 07:10:53.353022 2116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:53.353284 2116 log.cc:824] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:53.354028 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: No bootstrap required, opened a new log
I20260811 07:10:53.355607 2116 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } }
I20260811 07:10:53.355755 2116 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:53.355778 2116 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a0040dd5584f2d86bf474a2d817cdc, State: Initialized, Role: FOLLOWER
I20260811 07:10:53.355935 2116 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } }
I20260811 07:10:53.356022 2116 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:53.356047 2116 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:53.356077 2116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.356654 2116 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } }
I20260811 07:10:53.356745 2116 leader_election.cc:302] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0a0040dd5584f2d86bf474a2d817cdc; no voters:
I20260811 07:10:53.356886 2116 leader_election.cc:288] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:53.356954 2120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:53.357156 2120 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [term 1 LEADER]: Becoming Leader. State: Replica: c0a0040dd5584f2d86bf474a2d817cdc, State: Running, Role: LEADER
I20260811 07:10:53.357177 2116 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:53.357264 2120 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } }
I20260811 07:10:53.357731 2122 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } } }
I20260811 07:10:53.358026 2122 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:53.357918 2123 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0a0040dd5584f2d86bf474a2d817cdc. Latest consensus state: current_term: 1 leader_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a0040dd5584f2d86bf474a2d817cdc" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36717 } } }
I20260811 07:10:53.358250 2123 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:53.358393 2136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:53.358702 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 1985
I20260811 07:10:53.358810 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:53.358987 2136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:53.359045 4018 external_mini_cluster.cc:1428] Setting key eea176cb50c63352aab5c1b7b9f30b1f
I20260811 07:10:53.360843 2136 catalog_manager.cc:1347] Generated new cluster ID: 30eb498bcfeb4ae1a2e9596200564404
I20260811 07:10:53.360901 2136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:53.365327 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.359765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.3.236.190:36717)
0811 07:10:53.359965 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.359974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.360012 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:10:53.360128 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.360136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.360205 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.360400 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:53.360625 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.362004 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.363153 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.363756 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.363860 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.364625 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:53.364630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:53.364630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:53.364835 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:53.364874 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:53.364887 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:53.364949 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.365043 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.365119 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260811 07:10:53.369211 2136 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:53.369814 2136 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:53.379683 2136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: Generated new TSK 0
I20260811 07:10:53.379865 2136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:10:53.383738 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--builtin_ntp_servers=127.3.236.148:37341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:53.486142 2156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:53.486361 2156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:53.486383 2156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:53.487553 2156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:53.487598 2156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:53.487624 2156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:53.489195 2156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:53.489449 2156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:53.489768 2156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:10:53.495944 2156 server_base.cc:1034] running on GCE node
W20260811 07:10:53.495945 2165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.496078 2171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.495945 2167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.496476 2156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:53.496762 2156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:53.497951 2156 hybrid_clock.cc:648] HybridClock initialized: now 1786432253497929 us; error 37 us; skew 500 ppm
I20260811 07:10:53.499419 2156 webserver.cc:460] Webserver started at http://127.3.236.129:42955/ using document root <none> and password file <none>
I20260811 07:10:53.499704 2156 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:53.499778 2156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:53.499917 2156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:53.501439 2156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "02fa798044714d69ae507360bf5c9956"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "ae4a582d220f484d0e479dc93210ffcc"
server_key_iv: "806aa74c3dbec46af4fb52602e59ec37"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.501864 2156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "02fa798044714d69ae507360bf5c9956"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "ae4a582d220f484d0e479dc93210ffcc"
server_key_iv: "806aa74c3dbec46af4fb52602e59ec37"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.503640 2156 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 07:10:53.504637 2177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.504822 2156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:53.504935 2156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "02fa798044714d69ae507360bf5c9956"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "ae4a582d220f484d0e479dc93210ffcc"
server_key_iv: "806aa74c3dbec46af4fb52602e59ec37"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.505015 2156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:53.523931 2156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:53.524338 2156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:53.524507 2156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:53.524755 2156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:53.525120 2156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:53.525154 2156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.525177 2156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:53.525220 2156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.531250 2156 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:37025
I20260811 07:10:53.531315 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:37025 every 8 connection(s)
I20260811 07:10:53.531665 2156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:53.536806 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.532516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:58713 (local address 127.3.236.190:36717)
0811 07:10:53.532584 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.532587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.532865 (+ 278us) server_negotiation.cc:407] Connection header received
0811 07:10:53.533359 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.533360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.533373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.533427 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:53.534079 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.534557 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.535511 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.535692 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.535745 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.536590 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:53.536592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:53.536592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:53.536596 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:53.536609 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:53.536613 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:53.536661 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.536725 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.536749 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 07:10:53.537038 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.532551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36717 (local address 127.3.236.129:58713)
0811 07:10:53.532743 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.532778 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.533271 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.533486 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.533489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.533499 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:53.533986 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.533991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.534639 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.534640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.535439 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.535441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.535768 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.535769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.535882 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.536460 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:53.536460 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:53.536488 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:53.536512 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:53.536665 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:53.536667 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:53.536669 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.536716 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.536791 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260811 07:10:53.537602 2379 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36717
I20260811 07:10:53.537703 2379 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:53.537897 2379 heartbeater.cc:511] Master 127.3.236.190:36717 requested a full tablet report, sending...
I20260811 07:10:53.538503 2050 ts_manager.cc:194] Registered new tserver with Master: 02fa798044714d69ae507360bf5c9956 (127.3.236.129:37025)
I20260811 07:10:53.539173 2050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.129:58713
I20260811 07:10:53.540350 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 2156
I20260811 07:10:53.540467 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:53.540685 4018 external_mini_cluster.cc:1428] Setting key 8460720708256267246db7e3183ad5e6
I20260811 07:10:53.541600 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--builtin_ntp_servers=127.3.236.148:37341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:53.624550 2384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:53.624754 2384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:53.624776 2384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:53.625898 2384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:53.625942 2384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:53.625967 2384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:53.627313 2384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:53.627569 2384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:53.627877 2384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:53.633054 2401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.633159 2384 server_base.cc:1034] running on GCE node
W20260811 07:10:53.633044 2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.633044 2395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.633522 2384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:53.633765 2384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:53.634931 2384 hybrid_clock.cc:648] HybridClock initialized: now 1786432253634915 us; error 30 us; skew 500 ppm
I20260811 07:10:53.636238 2384 webserver.cc:460] Webserver started at http://127.3.236.130:33543/ using document root <none> and password file <none>
I20260811 07:10:53.636559 2384 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:53.636608 2384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:53.636770 2384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:53.637850 2384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "c095263716384f41a5062bcebb13c168"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "f98e8777541d2d1c035fd1aab61cb29b"
server_key_iv: "4f15acb8f4d05410c32a38ad3a012c6a"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.638214 2384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "c095263716384f41a5062bcebb13c168"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "f98e8777541d2d1c035fd1aab61cb29b"
server_key_iv: "4f15acb8f4d05410c32a38ad3a012c6a"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.639562 2384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:53.640326 2413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.640509 2384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:53.640604 2384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "c095263716384f41a5062bcebb13c168"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "f98e8777541d2d1c035fd1aab61cb29b"
server_key_iv: "4f15acb8f4d05410c32a38ad3a012c6a"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.640678 2384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:53.661432 2384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:53.661761 2384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:53.661916 2384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:53.662177 2384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:53.662559 2384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:53.662612 2384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.662649 2384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:53.662675 2384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.668533 2384 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:45237
I20260811 07:10:53.668614 2596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:45237 every 8 connection(s)
I20260811 07:10:53.668954 2384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:53.674002 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.669763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:32907 (local address 127.3.236.190:36717)
0811 07:10:53.669816 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.669818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.670061 (+ 243us) server_negotiation.cc:407] Connection header received
0811 07:10:53.670560 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.670561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.670571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.670631 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:53.671281 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.671777 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.672768 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.672962 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.673019 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.673768 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:53.673769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:53.673769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:53.673774 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:53.673788 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:53.673793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:53.673836 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.673917 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.673942 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:10:53.674235 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.669729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36717 (local address 127.3.236.130:32907)
0811 07:10:53.669925 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.669966 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.670458 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.670684 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.670688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.670696 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:53.671210 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.671214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.671860 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.671860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.672676 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.672678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.673042 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.673042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.673152 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.673636 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:53.673637 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:53.673665 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:53.673687 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:53.673846 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:53.673851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:53.673853 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.673905 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.673985 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260811 07:10:53.674778 2597 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36717
I20260811 07:10:53.674885 2597 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:53.675071 2597 heartbeater.cc:511] Master 127.3.236.190:36717 requested a full tablet report, sending...
I20260811 07:10:53.675431 2050 ts_manager.cc:194] Registered new tserver with Master: c095263716384f41a5062bcebb13c168 (127.3.236.130:45237)
I20260811 07:10:53.675796 2050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.130:32907
I20260811 07:10:53.677488 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 2384
I20260811 07:10:53.677589 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:10:53.677821 4018 external_mini_cluster.cc:1428] Setting key d3a4ad5d7e3707362975fb809c3698b1
I20260811 07:10:53.678748 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--builtin_ntp_servers=127.3.236.148:37341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:53.763401 2600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:53.763614 2600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:53.763638 2600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:53.764798 2600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:53.764837 2600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:53.764861 2600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:53.766170 2600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:37341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:53.766369 2600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:53.766626 2600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:53.771550 2608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:53.771595 2610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.771703 2600 server_base.cc:1034] running on GCE node
W20260811 07:10:53.771883 2607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:53.772132 2600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:53.772423 2600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:53.773584 2600 hybrid_clock.cc:648] HybridClock initialized: now 1786432253773571 us; error 30 us; skew 500 ppm
I20260811 07:10:53.774926 2600 webserver.cc:460] Webserver started at http://127.3.236.131:34779/ using document root <none> and password file <none>
I20260811 07:10:53.775187 2600 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:53.775259 2600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:53.775410 2600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:53.776579 2600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "e8f18abb9ae24a7fa875aab3514ca9a6"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "37db793ee795e3dc6c69339f3661caf3"
server_key_iv: "35c62b634f827ce9e8ae13d894e71737"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.776943 2600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "e8f18abb9ae24a7fa875aab3514ca9a6"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "37db793ee795e3dc6c69339f3661caf3"
server_key_iv: "35c62b634f827ce9e8ae13d894e71737"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.778245 2600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:53.779014 2620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.779181 2600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:53.779282 2600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "e8f18abb9ae24a7fa875aab3514ca9a6"
format_stamp: "Formatted at 2026-08-11 07:10:53 on dist-test-slave-2d19"
server_key: "37db793ee795e3dc6c69339f3661caf3"
server_key_iv: "35c62b634f827ce9e8ae13d894e71737"
server_key_version: "encryptionkey@0"
I20260811 07:10:53.779364 2600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:53.799683 2600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:53.800012 2600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:53.800166 2600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:53.800457 2600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:53.800804 2600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:53.800856 2600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.800915 2600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:53.800943 2600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:53.806713 2600 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:40235
I20260811 07:10:53.806789 2790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:40235 every 8 connection(s)
I20260811 07:10:53.807183 2600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:10:53.813380 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.808340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:42955 (local address 127.3.236.190:36717)
0811 07:10:53.808396 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.808399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.808592 (+ 193us) server_negotiation.cc:407] Connection header received
0811 07:10:53.809346 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.809348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.809362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.809425 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:53.810387 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.810955 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.811957 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.812172 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.812245 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.813103 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:53.813105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:53.813105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:53.813110 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:53.813126 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:53.813131 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:53.813185 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.813266 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.813299 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:10:53.813585 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.808094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36717 (local address 127.3.236.131:42955)
0811 07:10:53.808441 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.808485 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.809235 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.809498 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.809502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.809514 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:53.810284 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.810291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.811061 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.811062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.811853 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.811855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.812286 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.812287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.812430 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.812942 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:53.812943 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:53.812971 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:53.813013 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:53.813189 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:53.813194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:53.813197 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.813269 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.813369 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":101,"threads_started":1}
I20260811 07:10:53.814102 2791 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36717
I20260811 07:10:53.814188 2791 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:53.814344 2791 heartbeater.cc:511] Master 127.3.236.190:36717 requested a full tablet report, sending...
I20260811 07:10:53.814596 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 2600
I20260811 07:10:53.814705 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:10:53.814924 4018 external_mini_cluster.cc:1428] Setting key 1df15314cdbfc9f6464319b51c4be0d9
I20260811 07:10:53.815129 2050 ts_manager.cc:194] Registered new tserver with Master: e8f18abb9ae24a7fa875aab3514ca9a6 (127.3.236.131:40235)
I20260811 07:10:53.815562 2050 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.131:42955
I20260811 07:10:53.815807 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:53.819903 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.817285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.3.236.190:36717)
0811 07:10:53.817333 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.817336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.817457 (+ 121us) server_negotiation.cc:407] Connection header received
0811 07:10:53.817482 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.817482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.817495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.817548 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:53.817683 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.818149 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.818880 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.819063 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.819120 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.819684 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:53.819685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:53.819685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:53.819689 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:53.819701 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:53.819705 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:53.819747 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.819793 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.819813 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:10:53.821740 2046 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:53.823292 2046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:53.829713 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:37025 (local address 127.0.0.1:43078)
0811 07:10:53.825912 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.825923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.826039 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.826662 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.826663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.826665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.826713 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.826714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827577 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.827578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.828553 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.828555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.829389 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.829391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.829468 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.829483 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.829570 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.829611 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":92,"threads_started":1}
I20260811 07:10:53.829715 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.3.236.129:37025)
0811 07:10:53.826429 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.826434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.826442 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:10:53.826486 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.826487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.826498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.826584 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.826785 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827492 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.828645 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.829302 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.829380 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.829495 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.829587 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.829629 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":90,"threads_started":1}
I20260811 07:10:53.830098 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:40235 (local address 127.0.0.1:55964)
0811 07:10:53.825658 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.825703 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.825772 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.826310 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.826311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.826313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.826346 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.826348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827612 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.827613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.828900 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.828902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.829885 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.829886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.829947 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.829951 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.829985 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.830031 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":87,"thread_start_us":73,"threads_started":1}
I20260811 07:10:53.830133 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.3.236.131:40235)
0811 07:10:53.826162 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.826167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.826176 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:53.826212 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.826214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.826227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.826302 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.826449 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827511 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.828992 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.829813 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.829884 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.829967 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.830022 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.830055 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":86,"threads_started":1}
I20260811 07:10:53.830812 2697 tablet_service.cc:1467] Processing CreateTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b (DEFAULT_TABLE table=test-table [id=2fdbbba493964d06ac184a1740cb7703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:53.830821 2263 tablet_service.cc:1467] Processing CreateTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b (DEFAULT_TABLE table=test-table [id=2fdbbba493964d06ac184a1740cb7703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:53.831199 2263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3e5e3a102bf4c2f9f13ea483a162f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:53.831195 2697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3e5e3a102bf4c2f9f13ea483a162f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:53.831373 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:45237 (local address 127.0.0.1:42968)
0811 07:10:53.825892 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.825902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.825954 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.826176 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.826179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.826182 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.826251 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.826253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827612 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.827613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.828893 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.828896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.831131 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.831132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.831198 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.831202 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.831262 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.831302 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":56,"thread_start_us":117,"threads_started":1}
I20260811 07:10:53.831671 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.825630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.3.236.130:45237)
0811 07:10:53.825832 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.825838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.825983 (+ 145us) server_negotiation.cc:407] Connection header received
0811 07:10:53.826038 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.826041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.826059 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.826149 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.826340 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.827518 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.830254 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.831052 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.831129 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.831210 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.831540 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.831589 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260811 07:10:53.832939 2492 tablet_service.cc:1467] Processing CreateTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b (DEFAULT_TABLE table=test-table [id=2fdbbba493964d06ac184a1740cb7703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:53.833292 2492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3e5e3a102bf4c2f9f13ea483a162f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:53.834273 2813 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Bootstrap starting.
I20260811 07:10:53.834919 2813 tablet_bootstrap.cc:654] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:53.835227 2813 log.cc:824] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:53.835911 2813 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: No bootstrap required, opened a new log
I20260811 07:10:53.835964 2813 ts_tablet_manager.cc:1397] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:10:53.836393 2815 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: Bootstrap starting.
I20260811 07:10:53.836393 2819 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Bootstrap starting.
I20260811 07:10:53.837090 2819 tablet_bootstrap.cc:654] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:53.837100 2815 tablet_bootstrap.cc:654] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:53.837428 2819 log.cc:824] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:53.837427 2815 log.cc:824] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:53.838210 2815 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: No bootstrap required, opened a new log
I20260811 07:10:53.838286 2815 ts_tablet_manager.cc:1397] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:10:53.838501 2819 tablet_bootstrap.cc:492] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: No bootstrap required, opened a new log
I20260811 07:10:53.838572 2819 ts_tablet_manager.cc:1397] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:10:53.838506 2813 raft_consensus.cc:348] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.838682 2813 raft_consensus.cc:374] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:53.838745 2813 raft_consensus.cc:729] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f18abb9ae24a7fa875aab3514ca9a6, State: Initialized, Role: FOLLOWER
I20260811 07:10:53.838869 2813 consensus_queue.cc:260] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.839191 2791 heartbeater.cc:503] Master 127.3.236.190:36717 was elected leader, sending a full tablet report...
I20260811 07:10:53.839438 2813 ts_tablet_manager.cc:1428] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 07:10:53.841213 2819 raft_consensus.cc:348] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.841379 2819 raft_consensus.cc:374] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:53.841423 2819 raft_consensus.cc:729] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c095263716384f41a5062bcebb13c168, State: Initialized, Role: FOLLOWER
I20260811 07:10:53.841395 2815 raft_consensus.cc:348] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.841545 2815 raft_consensus.cc:374] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:53.841591 2815 raft_consensus.cc:729] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02fa798044714d69ae507360bf5c9956, State: Initialized, Role: FOLLOWER
I20260811 07:10:53.841552 2819 consensus_queue.cc:260] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.841698 2815 consensus_queue.cc:260] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.841816 2819 ts_tablet_manager.cc:1428] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:53.841863 2597 heartbeater.cc:503] Master 127.3.236.190:36717 was elected leader, sending a full tablet report...
I20260811 07:10:53.842206 2815 ts_tablet_manager.cc:1428] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 07:10:53.842310 2379 heartbeater.cc:503] Master 127.3.236.190:36717 was elected leader, sending a full tablet report...
I20260811 07:10:53.887329 2825 raft_consensus.cc:482] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:53.887459 2825 raft_consensus.cc:504] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.887890 2825 leader_election.cc:288] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c095263716384f41a5062bcebb13c168 (127.3.236.130:45237), 02fa798044714d69ae507360bf5c9956 (127.3.236.129:37025)
I20260811 07:10:53.890643 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.888114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:37357 (local address 127.3.236.129:37025)
0811 07:10:53.888159 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.888162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.888171 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:53.888243 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.888244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.888282 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.888334 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.888516 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.888996 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.889924 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.890299 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.890383 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.890392 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.890544 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.890584 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:10:53.890641 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.888044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:37025 (local address 127.3.236.131:37357)
0811 07:10:53.888116 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.888128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.888172 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.888349 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.888350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.888351 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.888422 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.888423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.889083 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.889083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.889827 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.889829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.890383 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.890384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.890455 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.890458 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.890524 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.890567 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 07:10:53.890803 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.888135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:46063 (local address 127.3.236.130:45237)
0811 07:10:53.888182 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.888184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.888353 (+ 169us) server_negotiation.cc:407] Connection header received
0811 07:10:53.888424 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.888425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.888438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.888504 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:53.888668 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.889225 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.890119 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.890461 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.890545 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.890553 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.890700 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.890738 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:10:53.890821 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.888051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:45237 (local address 127.3.236.131:46063)
0811 07:10:53.888248 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.888290 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:53.888368 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:53.888522 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:53.888523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:53.888525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:53.888598 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.888599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.889306 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.889307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.890030 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:53.890032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.890545 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.890546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:53.890618 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.890620 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:10:53.890692 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:10:53.890728 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":15,"thread_start_us":121,"threads_started":1}
I20260811 07:10:53.891148 2522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3e5e3a102bf4c2f9f13ea483a162f0b" candidate_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c095263716384f41a5062bcebb13c168" is_pre_election: true
I20260811 07:10:53.891166 2291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3e5e3a102bf4c2f9f13ea483a162f0b" candidate_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02fa798044714d69ae507360bf5c9956" is_pre_election: true
I20260811 07:10:53.891305 2522 raft_consensus.cc:2454] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f18abb9ae24a7fa875aab3514ca9a6 in term 0.
I20260811 07:10:53.891324 2291 raft_consensus.cc:2454] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f18abb9ae24a7fa875aab3514ca9a6 in term 0.
I20260811 07:10:53.891572 2623 leader_election.cc:302] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02fa798044714d69ae507360bf5c9956, e8f18abb9ae24a7fa875aab3514ca9a6; no voters:
I20260811 07:10:53.891691 2825 raft_consensus.cc:2781] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:53.891747 2825 raft_consensus.cc:482] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:53.891776 2825 raft_consensus.cc:3037] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.892549 2825 raft_consensus.cc:504] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.892709 2825 leader_election.cc:288] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [CANDIDATE]: Term 1 election: Requested vote from peers c095263716384f41a5062bcebb13c168 (127.3.236.130:45237), 02fa798044714d69ae507360bf5c9956 (127.3.236.129:37025)
I20260811 07:10:53.892884 2522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3e5e3a102bf4c2f9f13ea483a162f0b" candidate_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c095263716384f41a5062bcebb13c168"
I20260811 07:10:53.892889 2291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3e5e3a102bf4c2f9f13ea483a162f0b" candidate_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02fa798044714d69ae507360bf5c9956"
I20260811 07:10:53.892952 2522 raft_consensus.cc:3037] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.892971 2291 raft_consensus.cc:3037] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:53.893687 2522 raft_consensus.cc:2454] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f18abb9ae24a7fa875aab3514ca9a6 in term 1.
I20260811 07:10:53.893826 2291 raft_consensus.cc:2454] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f18abb9ae24a7fa875aab3514ca9a6 in term 1.
I20260811 07:10:53.894012 2621 leader_election.cc:302] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c095263716384f41a5062bcebb13c168, e8f18abb9ae24a7fa875aab3514ca9a6; no voters:
I20260811 07:10:53.894140 2825 raft_consensus.cc:2781] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:53.894263 2825 raft_consensus.cc:686] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 1 LEADER]: Becoming Leader. State: Replica: e8f18abb9ae24a7fa875aab3514ca9a6, State: Running, Role: LEADER
I20260811 07:10:53.894356 2825 consensus_queue.cc:237] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } }
I20260811 07:10:53.895196 2050 catalog_manager.cc:5495] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8f18abb9ae24a7fa875aab3514ca9a6 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8f18abb9ae24a7fa875aab3514ca9a6" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 40235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260811 07:10:53.904764 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.902456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.3.236.131:40235)
0811 07:10:53.902504 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:53.902506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:53.902516 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:10:53.902600 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:53.902601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:53.902617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:53.902690 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:53.902794 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.903243 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.903979 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:53.904163 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:53.904235 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:53.904429 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:53.904523 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:53.904606 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:53.904647 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 07:10:53.904665 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:10:53.906440 2522 raft_consensus.cc:1261] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 1 FOLLOWER]: Refusing update from remote peer e8f18abb9ae24a7fa875aab3514ca9a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:53.906447 2291 raft_consensus.cc:1261] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 1 FOLLOWER]: Refusing update from remote peer e8f18abb9ae24a7fa875aab3514ca9a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:53.906771 2825 consensus_queue.cc:1035] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02fa798044714d69ae507360bf5c9956" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:53.906911 2830 consensus_queue.cc:1035] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c095263716384f41a5062bcebb13c168" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:53.909698 2835 mvcc.cc:205] Tried to move back new op lower bound from 7317226511998320640 to 7317226511951937536. Current Snapshot: MvccSnapshot[applied={T|T < 7317226511998320640}]
I20260811 07:10:53.911957 2046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:53.912055 2046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:53.912815 2046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0a0040dd5584f2d86bf474a2d817cdc: Sending DeleteTablet for 3 replicas of tablet c3e5e3a102bf4c2f9f13ea483a162f0b
I20260811 07:10:53.913170 2263 tablet_service.cc:1514] Processing DeleteTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='slave'} at 127.0.0.1:43078
I20260811 07:10:53.913213 2697 tablet_service.cc:1514] Processing DeleteTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='slave'} at 127.0.0.1:55964
I20260811 07:10:53.913393 2492 tablet_service.cc:1514] Processing DeleteTablet for tablet c3e5e3a102bf4c2f9f13ea483a162f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:53 UTC) from {username='slave'} at 127.0.0.1:42968
I20260811 07:10:53.913415 2853 tablet_replica.cc:330] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956: stopping tablet replica
I20260811 07:10:53.913532 2853 raft_consensus.cc:2229] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:53.913625 2855 tablet_replica.cc:330] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: stopping tablet replica
I20260811 07:10:53.913717 2853 raft_consensus.cc:2258] T c3e5e3a102bf4c2f9f13ea483a162f0b P 02fa798044714d69ae507360bf5c9956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:53.913743 2855 raft_consensus.cc:2229] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:53.913792 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 2156
I20260811 07:10:53.916690 2855 raft_consensus.cc:2258] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:53.926702 2855 ts_tablet_manager.cc:1905] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:53.929332 2855 ts_tablet_manager.cc:1918] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:53.929437 2855 log.cc:1196] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/c3e5e3a102bf4c2f9f13ea483a162f0b
I20260811 07:10:53.929780 2855 ts_tablet_manager.cc:1939] T c3e5e3a102bf4c2f9f13ea483a162f0b P c095263716384f41a5062bcebb13c168: Deleting consensus metadata
I20260811 07:10:53.930523 2854 tablet_replica.cc:330] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: stopping tablet replica
I20260811 07:10:53.930663 2854 raft_consensus.cc:2229] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:10:53.930878 2854 raft_consensus.cc:2258] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:53.932612 2011 catalog_manager.cc:4841] TS c095263716384f41a5062bcebb13c168 (127.3.236.130:45237): tablet c3e5e3a102bf4c2f9f13ea483a162f0b (table test-table [id=2fdbbba493964d06ac184a1740cb7703]) successfully deleted
I20260811 07:10:53.933398 2854 ts_tablet_manager.cc:1905] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:53.935395 2854 ts_tablet_manager.cc:1918] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:53.935482 2854 log.cc:1196] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/c3e5e3a102bf4c2f9f13ea483a162f0b
I20260811 07:10:53.935791 2854 ts_tablet_manager.cc:1939] T c3e5e3a102bf4c2f9f13ea483a162f0b P e8f18abb9ae24a7fa875aab3514ca9a6: Deleting consensus metadata
I20260811 07:10:53.936339 2015 catalog_manager.cc:4841] TS e8f18abb9ae24a7fa875aab3514ca9a6 (127.3.236.131:40235): tablet c3e5e3a102bf4c2f9f13ea483a162f0b (table test-table [id=2fdbbba493964d06ac184a1740cb7703]) successfully deleted
W20260811 07:10:53.938181 2014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:37025 (error 108)
I20260811 07:10:53.938422 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 2384
I20260811 07:10:53.940541 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:53.938578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:37025 (local address 127.0.0.1:43084)
0811 07:10:53.940439 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:53.940496 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:37025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1833}
W20260811 07:10:53.946287 2014 catalog_manager.cc:4584] TS 02fa798044714d69ae507360bf5c9956 (127.3.236.129:37025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3e5e3a102bf4c2f9f13ea483a162f0b: Network error: Client connection negotiation failed: client connection to 127.3.236.129:37025: connect: Connection refused (error 111)
I20260811 07:10:53.956362 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 2600
I20260811 07:10:53.965770 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 1985
2026-08-11T07:10:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (775 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:10:54 dist-test-slave-2d19 krb5kdc[2861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:10:54 dist-test-slave-2d19 krb5kdc[2861](info): set up 2 sockets
Aug 11 07:10:54 dist-test-slave-2d19 krb5kdc[2861](info): commencing operation
krb5kdc: starting...
W20260811 07:10:55.264211 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:10:55.390486 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35375
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:55.474857 2877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:55.475065 2877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:55.475088 2877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:55.476315 2877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:55.476357 2877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:55.476372 2877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:55.476408 2877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:55.476438 2877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:55.477891 2877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:55.478147 2877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:55.478384 2877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:55.483281 2890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:55.483307 2893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:55.483530 2877 server_base.cc:1034] running on GCE node
W20260811 07:10:55.483281 2891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:55.483963 2877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:55.484218 2877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:55.485424 2877 hybrid_clock.cc:648] HybridClock initialized: now 1786432255485413 us; error 28 us; skew 500 ppm
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:55.487908 2877 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:10:55.488430 2877 webserver.cc:460] Webserver started at http://127.3.236.190:39577/ using document root <none> and password file <none>
I20260811 07:10:55.488663 2877 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:55.488729 2877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:55.488867 2877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:55.489938 2877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "f52e94bbd425468e98b9ad8016be6862"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "7ed943cc4c64fb95c5bb2c96a31303e5"
server_key_iv: "e205098bc26bed26ab97fecb300eb57f"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.490311 2877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "f52e94bbd425468e98b9ad8016be6862"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "7ed943cc4c64fb95c5bb2c96a31303e5"
server_key_iv: "e205098bc26bed26ab97fecb300eb57f"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.491755 2877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:55.492517 2905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.492686 2877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:10:55.492786 2877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "f52e94bbd425468e98b9ad8016be6862"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "7ed943cc4c64fb95c5bb2c96a31303e5"
server_key_iv: "e205098bc26bed26ab97fecb300eb57f"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.492863 2877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:55.514899 2877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:55.516959 2877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:55.517127 2877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:55.521342 2877 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35375
I20260811 07:10:55.521394 2983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35375 every 8 connection(s)
I20260811 07:10:55.521764 2877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:55.522656 2984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:55.525096 2984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: Bootstrap starting.
I20260811 07:10:55.525651 2984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:55.525893 2984 log.cc:824] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:55.526238 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 2877
I20260811 07:10:55.526382 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:55.526616 4018 external_mini_cluster.cc:1428] Setting key 54f369e6664ed1bfef9106bc893929cf
I20260811 07:10:55.526734 2984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: No bootstrap required, opened a new log
I20260811 07:10:55.528753 2984 raft_consensus.cc:348] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } }
I20260811 07:10:55.528913 2984 raft_consensus.cc:374] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:55.528963 2984 raft_consensus.cc:729] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52e94bbd425468e98b9ad8016be6862, State: Initialized, Role: FOLLOWER
I20260811 07:10:55.529093 2984 consensus_queue.cc:260] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } }
I20260811 07:10:55.529206 2984 raft_consensus.cc:388] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:55.529250 2984 raft_consensus.cc:482] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:55.529299 2984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:55.529848 2984 raft_consensus.cc:504] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } }
I20260811 07:10:55.529968 2984 leader_election.cc:302] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f52e94bbd425468e98b9ad8016be6862; no voters:
I20260811 07:10:55.530164 2984 leader_election.cc:288] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:55.530189 2991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:55.530311 2991 raft_consensus.cc:686] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [term 1 LEADER]: Becoming Leader. State: Replica: f52e94bbd425468e98b9ad8016be6862, State: Running, Role: LEADER
I20260811 07:10:55.530438 2984 sys_catalog.cc:564] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:55.530474 2991 consensus_queue.cc:237] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } }
I20260811 07:10:55.531014 2991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f52e94bbd425468e98b9ad8016be6862. Latest consensus state: current_term: 1 leader_uuid: "f52e94bbd425468e98b9ad8016be6862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } } }
I20260811 07:10:55.531019 2993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f52e94bbd425468e98b9ad8016be6862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e94bbd425468e98b9ad8016be6862" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35375 } } }
I20260811 07:10:55.531170 2991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:55.531255 2993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:55.531478 3009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:55.532094 3009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:55.534374 3009 catalog_manager.cc:1347] Generated new cluster ID: 588d11751a83416b9c42a306275c4876
I20260811 07:10:55.534430 3009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:55.539031 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:55.527144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53954 (local address 127.3.236.190:35375)
0811 07:10:55.527327 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:55.527337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:55.527373 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:10:55.528001 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:55.528011 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:55.528060 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:55.528204 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:55.528854 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.530340 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.531450 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.532107 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.532221 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:55.535249 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:55.535252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:55.535253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:55.535264 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:55.537179 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.537538 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.537541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.537541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.537644 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.537809 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.537809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.537810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.538085 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:55.538236 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:55.538697 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 07:10:55.538823 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260811 07:10:55.565606 3009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:55.566395 3009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:55.569178 3009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: Generated new TSK 0
I20260811 07:10:55.569372 3009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:10:55.599174 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:55.682250 3033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:55.682451 3033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:55.682474 3033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:55.683543 3033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:55.683578 3033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:55.683604 3033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:55.685148 3033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:55.685350 3033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:55.685527 3033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:55.690776 3040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:55.690936 3033 server_base.cc:1034] running on GCE node
W20260811 07:10:55.690793 3044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:55.690781 3042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:55.691278 3033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:55.691538 3033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:55.692750 3033 hybrid_clock.cc:648] HybridClock initialized: now 1786432255692738 us; error 29 us; skew 500 ppm
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:55.695312 3033 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:10:55.695804 3033 webserver.cc:460] Webserver started at http://127.3.236.129:38791/ using document root <none> and password file <none>
I20260811 07:10:55.696034 3033 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:55.696105 3033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:55.696290 3033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:55.697387 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "a12ef9c7320a49fc91fde9f922b4743f"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "e730cab8c50a41de26edd43ca5c52b01"
server_key_iv: "56cfe63be65e272a98946181adf42d89"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.697758 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "a12ef9c7320a49fc91fde9f922b4743f"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "e730cab8c50a41de26edd43ca5c52b01"
server_key_iv: "56cfe63be65e272a98946181adf42d89"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.699100 3033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:10:55.699790 3062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.699950 3033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:55.700053 3033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "a12ef9c7320a49fc91fde9f922b4743f"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "e730cab8c50a41de26edd43ca5c52b01"
server_key_iv: "56cfe63be65e272a98946181adf42d89"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.700133 3033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:55.709805 3033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:55.711714 3033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:55.711889 3033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:55.712131 3033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:55.712587 3033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:55.712644 3033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.712678 3033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:55.712706 3033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.718966 3033 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:44505
I20260811 07:10:55.719019 3258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:44505 every 8 connection(s)
I20260811 07:10:55.719324 3033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:55.724560 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 3033
I20260811 07:10:55.724697 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:55.724936 4018 external_mini_cluster.cc:1428] Setting key cd1ae092ef206bf40cc7fe168fef012b
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:55.730546 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:55.720303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:42121 (local address 127.3.236.190:35375)
0811 07:10:55.720351 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:55.720353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:55.720578 (+ 225us) server_negotiation.cc:407] Connection header received
0811 07:10:55.721149 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:55.721151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:55.721163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:55.721233 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:55.722204 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.723180 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.724203 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.724547 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.724630 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:55.726947 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:55.726949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:55.726949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:55.726957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:55.728791 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.729343 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.729344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.729344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.729402 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.729735 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.729736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.729736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.729846 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:55.729903 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:55.730232 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 07:10:55.730448 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:10:55.730733 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:55.720242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35375 (local address 127.3.236.129:42121)
0811 07:10:55.720457 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:55.720495 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:55.721040 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:55.721305 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:55.721310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:55.721910 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:55.722102 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:55.722106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.723288 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.723289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:55.724107 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:55.724109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.724674 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.724676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:55.724841 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:55.725464 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:55.725473 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:55.726846 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:55.728902 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:55.728912 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:55.728914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:55.729249 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:55.729490 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:55.729491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:55.729491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:55.729654 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:55.729913 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:55.729920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:55.730090 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 07:10:55.730207 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 07:10:55.730395 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
I20260811 07:10:55.731277 3263 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35375
I20260811 07:10:55.731390 3263 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:55.731616 3263 heartbeater.cc:511] Master 127.3.236.190:35375 requested a full tablet report, sending...
I20260811 07:10:55.732316 2930 ts_manager.cc:194] Registered new tserver with Master: a12ef9c7320a49fc91fde9f922b4743f (127.3.236.129:44505)
I20260811 07:10:55.733314 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:42121
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:10:55.755405 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:55.841199 3272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:55.841409 3272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:55.841432 3272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:55.842563 3272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:55.842605 3272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:55.842655 3272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:55.844074 3272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:55.844333 3272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:55.844576 3272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:55.849684 3283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:55.849677 3277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:55.849684 3278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:55.849990 3272 server_base.cc:1034] running on GCE node
I20260811 07:10:55.850193 3272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:55.850395 3272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:55.851557 3272 hybrid_clock.cc:648] HybridClock initialized: now 1786432255851546 us; error 37 us; skew 500 ppm
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:55.854244 3272 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:10:55.854729 3272 webserver.cc:460] Webserver started at http://127.3.236.130:41747/ using document root <none> and password file <none>
I20260811 07:10:55.854952 3272 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:55.855021 3272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:55.855177 3272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:55.856230 3272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "9ffc0907be8c498598cbb7afbeca249d"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "653bef35e345ca61bbc2e616b41e4e2d"
server_key_iv: "dcda43dd99c10809a492d8822bcd73fb"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.856670 3272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "9ffc0907be8c498598cbb7afbeca249d"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "653bef35e345ca61bbc2e616b41e4e2d"
server_key_iv: "dcda43dd99c10809a492d8822bcd73fb"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.858017 3272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:55.858747 3293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.858914 3272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:55.859015 3272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "9ffc0907be8c498598cbb7afbeca249d"
format_stamp: "Formatted at 2026-08-11 07:10:55 on dist-test-slave-2d19"
server_key: "653bef35e345ca61bbc2e616b41e4e2d"
server_key_iv: "dcda43dd99c10809a492d8822bcd73fb"
server_key_version: "encryptionkey@0"
I20260811 07:10:55.859092 3272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:55.898751 3272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:55.901247 3272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:55.901417 3272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:55.901695 3272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:55.902083 3272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:55.902153 3272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.902199 3272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:55.902235 3272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:55.908439 3272 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:46805
I20260811 07:10:55.908499 3463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:46805 every 8 connection(s)
I20260811 07:10:55.908825 3272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:55.911690 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 3272
I20260811 07:10:55.911821 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:10:55.912087 4018 external_mini_cluster.cc:1428] Setting key 4f11c51fc96fe04b91e8cc3c9e346407
Aug 11 07:10:55 dist-test-slave-2d19 krb5kdc[2861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432255, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:55.918900 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:55.909521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:42135 (local address 127.3.236.190:35375)
0811 07:10:55.909570 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:55.909571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:55.909794 (+ 223us) server_negotiation.cc:407] Connection header received
0811 07:10:55.910283 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:55.910284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:55.910294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:55.910349 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:55.911275 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.911776 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.913040 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.913225 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.913282 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:55.916070 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:55.916072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:55.916072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:55.916079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:55.917485 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.917908 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.917909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.917910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.917973 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:55.918235 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:55.918236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:55.918236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:55.918359 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:55.918435 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:55.918716 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 07:10:55.918804 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:10:55.919166 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:55.909485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35375 (local address 127.3.236.130:42135)
0811 07:10:55.909673 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:55.909717 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:55.910200 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:55.910412 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:55.910415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:55.910991 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:55.911190 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:55.911196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.911912 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.911914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:55.912955 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:55.912958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:55.913308 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:55.913310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:55.913451 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:55.913980 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:55.913991 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:55.915949 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:55.917586 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:55.917595 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:55.917597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:55.917828 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:55.918047 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:55.918048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:55.918049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:55.918167 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:55.918426 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:55.918434 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:55.918580 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 07:10:55.918690 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 07:10:55.918857 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260811 07:10:55.919684 3464 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35375
I20260811 07:10:55.919800 3464 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:55.919993 3464 heartbeater.cc:511] Master 127.3.236.190:35375 requested a full tablet report, sending...
I20260811 07:10:55.920445 2930 ts_manager.cc:194] Registered new tserver with Master: 9ffc0907be8c498598cbb7afbeca249d (127.3.236.130:46805)
I20260811 07:10:55.920836 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:42135
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:10:55.942549 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:56.026249 3470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:56.026473 3470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:56.026499 3470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:56.027717 3470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:56.027760 3470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:56.027787 3470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:56.029243 3470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:56.029487 3470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:56.029727 3470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:56.035174 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.035166 3477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.035167 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.035480 3470 server_base.cc:1034] running on GCE node
I20260811 07:10:56.035691 3470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:56.035930 3470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:56.037070 3470 hybrid_clock.cc:648] HybridClock initialized: now 1786432256037050 us; error 35 us; skew 500 ppm
Aug 11 07:10:56 dist-test-slave-2d19 krb5kdc[2861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:56.039499 3470 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:10:56.039883 3470 webserver.cc:460] Webserver started at http://127.3.236.131:39293/ using document root <none> and password file <none>
I20260811 07:10:56.040141 3470 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:56.040187 3470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:56.040333 3470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:56.041325 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "8c8f36b356774ab293eb651c1ff471e1"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "be777f2f2b8268d96e14d6597cfacd6d"
server_key_iv: "e0fe1d6e7848f563b0217500745950a8"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.041623 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "8c8f36b356774ab293eb651c1ff471e1"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "be777f2f2b8268d96e14d6597cfacd6d"
server_key_iv: "e0fe1d6e7848f563b0217500745950a8"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.042860 3470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:10:56.043526 3492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.043692 3470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:56.043794 3470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "8c8f36b356774ab293eb651c1ff471e1"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "be777f2f2b8268d96e14d6597cfacd6d"
server_key_iv: "e0fe1d6e7848f563b0217500745950a8"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.043866 3470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:56.055445 3470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:56.057433 3470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:56.057595 3470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:56.057827 3470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:56.058161 3470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:56.058216 3470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.058260 3470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:56.058286 3470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.064393 3470 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:45093
I20260811 07:10:56.064499 3657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:45093 every 8 connection(s)
I20260811 07:10:56.064792 3470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:10:56.068168 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 3470
I20260811 07:10:56.068291 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:10:56.068468 4018 external_mini_cluster.cc:1428] Setting key 945d550501a842f3443efc7356d0e747
Aug 11 07:10:56 dist-test-slave-2d19 krb5kdc[2861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:56.074445 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.065543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:60691 (local address 127.3.236.190:35375)
0811 07:10:56.065592 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.065594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.065806 (+ 212us) server_negotiation.cc:407] Connection header received
0811 07:10:56.066302 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.066303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.066313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.066373 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.067267 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.067772 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.069078 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.069393 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.069482 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.071728 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.071730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.071730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:56.071738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.073056 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:56.073461 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:56.073462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:56.073463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:56.073535 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:56.073788 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:56.073789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:56.073790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:56.073910 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.073978 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.074264 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.074348 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:10:56.074645 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.065510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35375 (local address 127.3.236.131:60691)
0811 07:10:56.065671 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.065714 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.066219 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.066427 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.066430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.066985 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:56.067190 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.067195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.067853 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.067854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.069005 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.069008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.069494 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.069495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.069676 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.070254 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:56.070264 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:56.071626 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:56.073142 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:56.073150 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:56.073153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:56.073382 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:56.073602 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:56.073602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:56.073603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:56.073715 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:56.073981 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:56.073989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:56.074134 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.074241 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.074408 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260811 07:10:56.075290 3658 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35375
I20260811 07:10:56.075395 3658 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:56.075547 3658 heartbeater.cc:511] Master 127.3.236.190:35375 requested a full tablet report, sending...
I20260811 07:10:56.075932 2930 ts_manager.cc:194] Registered new tserver with Master: 8c8f36b356774ab293eb651c1ff471e1 (127.3.236.131:45093)
I20260811 07:10:56.076354 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:60691
I20260811 07:10:56.080755 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:56.088320 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.082091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.3.236.190:35375)
0811 07:10:56.082159 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.082161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.082234 (+ 73us) server_negotiation.cc:407] Connection header received
0811 07:10:56.082271 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.082272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.082280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.082321 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.082839 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.083317 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.084011 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.084185 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.084235 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.085583 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.085584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.085585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:56.085590 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.087061 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:56.087462 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:56.087463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:56.087463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:56.087514 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:56.087803 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:56.087803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:56.087803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:56.087926 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.088024 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.088129 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.088185 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:10:56.089875 2930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:56.091871 2930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:56.098441 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46805 (local address 127.0.0.1:42634)
0811 07:10:56.094432 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.094482 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.094548 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.094886 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.094889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.094892 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.094991 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.094993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096441 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.096442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.097445 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.097448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098115 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.098196 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098209 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.098308 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.098362 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":66,"threads_started":1}
I20260811 07:10:56.098456 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.3.236.130:46805)
0811 07:10:56.094713 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.094719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.094729 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:10:56.094770 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.094772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.094786 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.094881 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.095108 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096321 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.097531 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098029 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098114 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098177 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.098329 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.098370 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260811 07:10:56.098872 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44505 (local address 127.0.0.1:51080)
0811 07:10:56.094704 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.094715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.094770 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.095352 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.095353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.095355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.095392 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.095393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096712 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.096713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.097761 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.097764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098638 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.098697 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098701 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.098757 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.098796 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":87,"threads_started":1}
I20260811 07:10:56.098879 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.3.236.129:44505)
0811 07:10:56.095165 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.095169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.095180 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:10:56.095219 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.095221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.095236 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.095348 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.095491 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096592 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.097849 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098571 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098634 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098687 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.098767 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.098806 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":97,"threads_started":1}
I20260811 07:10:56.098878 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.3.236.131:45093)
0811 07:10:56.094870 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.094874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.094882 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:10:56.094917 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.094918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.094930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.095013 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.095181 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096008 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.097579 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098271 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098339 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098391 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.098756 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.098800 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":91,"threads_started":1}
I20260811 07:10:56.098682 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.094238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:45093 (local address 127.0.0.1:55718)
0811 07:10:56.094657 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.094667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.094705 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.095047 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.095048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.095050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.095097 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.095099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.096118 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.096119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.097468 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.097471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.098428 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.098429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.098489 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.098506 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.098560 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.098609 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260811 07:10:56.099622 3370 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf55783c5346eea5b8da530b331215 (DEFAULT_TABLE table=test-table [id=df6ad6ac46124aa0a62192bf0163509a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.099742 3140 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf55783c5346eea5b8da530b331215 (DEFAULT_TABLE table=test-table [id=df6ad6ac46124aa0a62192bf0163509a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.099993 3370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf55783c5346eea5b8da530b331215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.100032 3140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf55783c5346eea5b8da530b331215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.102697 3684 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Bootstrap starting.
I20260811 07:10:56.102945 3682 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: Bootstrap starting.
I20260811 07:10:56.103263 3558 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf55783c5346eea5b8da530b331215 (DEFAULT_TABLE table=test-table [id=df6ad6ac46124aa0a62192bf0163509a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.103502 3684 tablet_bootstrap.cc:654] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.103609 3558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf55783c5346eea5b8da530b331215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.103730 3682 tablet_bootstrap.cc:654] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.104140 3684 log.cc:824] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.104203 3682 log.cc:824] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.104910 3682 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: No bootstrap required, opened a new log
I20260811 07:10:56.104991 3682 ts_tablet_manager.cc:1397] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:10:56.106262 3687 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Bootstrap starting.
I20260811 07:10:56.106389 3682 raft_consensus.cc:348] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.106573 3682 raft_consensus.cc:374] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.106642 3682 raft_consensus.cc:729] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a12ef9c7320a49fc91fde9f922b4743f, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.106818 3682 consensus_queue.cc:260] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.107004 3684 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: No bootstrap required, opened a new log
I20260811 07:10:56.107080 3684 ts_tablet_manager.cc:1397] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:10:56.107115 3682 ts_tablet_manager.cc:1428] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:10:56.107239 3263 heartbeater.cc:503] Master 127.3.236.190:35375 was elected leader, sending a full tablet report...
I20260811 07:10:56.108856 3684 raft_consensus.cc:348] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.109019 3684 raft_consensus.cc:374] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.109068 3684 raft_consensus.cc:729] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffc0907be8c498598cbb7afbeca249d, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.109181 3684 consensus_queue.cc:260] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.109474 3684 ts_tablet_manager.cc:1428] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 07:10:56.109510 3464 heartbeater.cc:503] Master 127.3.236.190:35375 was elected leader, sending a full tablet report...
I20260811 07:10:56.109818 3687 tablet_bootstrap.cc:654] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.110477 3687 log.cc:824] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.111284 3687 tablet_bootstrap.cc:492] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: No bootstrap required, opened a new log
I20260811 07:10:56.111349 3687 ts_tablet_manager.cc:1397] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 07:10:56.112733 3687 raft_consensus.cc:348] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.112864 3687 raft_consensus.cc:374] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.112886 3687 raft_consensus.cc:729] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8f36b356774ab293eb651c1ff471e1, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.112987 3687 consensus_queue.cc:260] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.113224 3687 ts_tablet_manager.cc:1428] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:56.113272 3658 heartbeater.cc:503] Master 127.3.236.190:35375 was elected leader, sending a full tablet report...
I20260811 07:10:56.124325 3691 raft_consensus.cc:482] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:56.124431 3691 raft_consensus.cc:504] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.124823 3691 leader_election.cc:288] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ffc0907be8c498598cbb7afbeca249d (127.3.236.130:46805), a12ef9c7320a49fc91fde9f922b4743f (127.3.236.129:44505)
I20260811 07:10:56.127843 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.125121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:46997 (local address 127.3.236.129:44505)
0811 07:10:56.125170 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.125172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.125184 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:10:56.125218 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.125219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.125230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.125304 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.125516 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.126063 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127063 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.127477 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127536 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.127543 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.127728 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.127777 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:10:56.127835 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.125028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44505 (local address 127.3.236.131:46997)
0811 07:10:56.125099 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.125112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.125158 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.125353 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.125354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.125357 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.125439 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.125441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.126167 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.126168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.126944 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.126946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.127562 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.127651 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.127655 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.127711 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.127745 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 07:10:56.127916 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.125133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:35273 (local address 127.3.236.130:46805)
0811 07:10:56.125177 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.125179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.125338 (+ 159us) server_negotiation.cc:407] Connection header received
0811 07:10:56.125378 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.125379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.125392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.125469 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.125681 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.126187 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127134 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.127557 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127634 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.127643 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.127802 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.127843 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:10:56.127957 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.125028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:46805 (local address 127.3.236.131:35273)
0811 07:10:56.125250 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.125261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.125323 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.125499 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.125500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.125512 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.125589 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.125591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.126271 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.126272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.127041 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.127044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.127647 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.127648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.127723 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.127727 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.127784 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.127822 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":40,"thread_start_us":121,"threads_started":1}
I20260811 07:10:56.128302 3181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf55783c5346eea5b8da530b331215" candidate_uuid: "8c8f36b356774ab293eb651c1ff471e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a12ef9c7320a49fc91fde9f922b4743f" is_pre_election: true
I20260811 07:10:56.128363 3394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf55783c5346eea5b8da530b331215" candidate_uuid: "8c8f36b356774ab293eb651c1ff471e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffc0907be8c498598cbb7afbeca249d" is_pre_election: true
I20260811 07:10:56.128482 3394 raft_consensus.cc:2454] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8f36b356774ab293eb651c1ff471e1 in term 0.
I20260811 07:10:56.128472 3181 raft_consensus.cc:2454] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8f36b356774ab293eb651c1ff471e1 in term 0.
I20260811 07:10:56.128753 3495 leader_election.cc:302] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c8f36b356774ab293eb651c1ff471e1, a12ef9c7320a49fc91fde9f922b4743f; no voters:
I20260811 07:10:56.128885 3691 raft_consensus.cc:2781] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:56.128973 3691 raft_consensus.cc:482] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:56.129006 3691 raft_consensus.cc:3037] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.129717 3691 raft_consensus.cc:504] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.129868 3691 leader_election.cc:288] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [CANDIDATE]: Term 1 election: Requested vote from peers 9ffc0907be8c498598cbb7afbeca249d (127.3.236.130:46805), a12ef9c7320a49fc91fde9f922b4743f (127.3.236.129:44505)
I20260811 07:10:56.130015 3181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf55783c5346eea5b8da530b331215" candidate_uuid: "8c8f36b356774ab293eb651c1ff471e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a12ef9c7320a49fc91fde9f922b4743f"
I20260811 07:10:56.130113 3394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf55783c5346eea5b8da530b331215" candidate_uuid: "8c8f36b356774ab293eb651c1ff471e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffc0907be8c498598cbb7afbeca249d"
I20260811 07:10:56.130102 3181 raft_consensus.cc:3037] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.130190 3394 raft_consensus.cc:3037] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.131114 3181 raft_consensus.cc:2454] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8f36b356774ab293eb651c1ff471e1 in term 1.
I20260811 07:10:56.131114 3394 raft_consensus.cc:2454] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8f36b356774ab293eb651c1ff471e1 in term 1.
I20260811 07:10:56.131299 3497 leader_election.cc:302] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c8f36b356774ab293eb651c1ff471e1, 9ffc0907be8c498598cbb7afbeca249d; no voters:
I20260811 07:10:56.131397 3691 raft_consensus.cc:2781] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:56.131549 3691 raft_consensus.cc:686] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8f36b356774ab293eb651c1ff471e1, State: Running, Role: LEADER
I20260811 07:10:56.131642 3691 consensus_queue.cc:237] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } }
I20260811 07:10:56.132532 2930 catalog_manager.cc:5495] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8f36b356774ab293eb651c1ff471e1 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "8c8f36b356774ab293eb651c1ff471e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8f36b356774ab293eb651c1ff471e1" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 45093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 } health_report { overall_health: UNKNOWN } } }
I20260811 07:10:56.145081 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.142639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.3.236.131:45093)
0811 07:10:56.142712 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.142714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.142723 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:56.142802 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.142802 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.142826 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.142895 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:56.143044 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.143504 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.144319 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.144520 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.144596 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.144733 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:56.144826 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:56.144934 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.144966 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.144977 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:10:56.146916 3394 raft_consensus.cc:1261] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 1 FOLLOWER]: Refusing update from remote peer 8c8f36b356774ab293eb651c1ff471e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:56.146915 3181 raft_consensus.cc:1261] T fdbf55783c5346eea5b8da530b331215 P a12ef9c7320a49fc91fde9f922b4743f [term 1 FOLLOWER]: Refusing update from remote peer 8c8f36b356774ab293eb651c1ff471e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:56.147280 3691 consensus_queue.cc:1035] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ffc0907be8c498598cbb7afbeca249d" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 46805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:56.147423 3693 consensus_queue.cc:1035] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a12ef9c7320a49fc91fde9f922b4743f" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:56.149905 3697 mvcc.cc:205] Tried to move back new op lower bound from 7317226521174867968 to 7317226521115951104. Current Snapshot: MvccSnapshot[applied={T|T < 7317226521174867968}]
I20260811 07:10:56.150244 3699 mvcc.cc:205] Tried to move back new op lower bound from 7317226521174867968 to 7317226521115951104. Current Snapshot: MvccSnapshot[applied={T|T < 7317226521174867968}]
I20260811 07:10:56.150555 3698 mvcc.cc:205] Tried to move back new op lower bound from 7317226521174867968 to 7317226521115951104. Current Snapshot: MvccSnapshot[applied={T|T < 7317226521174867968}]
I20260811 07:10:56.152798 2928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:56.152910 2928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:56.153709 2928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f52e94bbd425468e98b9ad8016be6862: Sending DeleteTablet for 3 replicas of tablet fdbf55783c5346eea5b8da530b331215
I20260811 07:10:56.154073 3140 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf55783c5346eea5b8da530b331215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:51080
I20260811 07:10:56.154115 3370 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf55783c5346eea5b8da530b331215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:42634
I20260811 07:10:56.154306 3558 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf55783c5346eea5b8da530b331215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:55718
I20260811 07:10:56.154343 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 3033
I20260811 07:10:56.154594 3710 tablet_replica.cc:330] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: stopping tablet replica
I20260811 07:10:56.154719 3711 tablet_replica.cc:330] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: stopping tablet replica
I20260811 07:10:56.154845 3711 raft_consensus.cc:2229] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:10:56.154992 3710 raft_consensus.cc:2229] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:56.155128 3710 raft_consensus.cc:2258] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:56.155328 3711 raft_consensus.cc:2258] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:56.161388 3710 ts_tablet_manager.cc:1905] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:56.161388 3711 ts_tablet_manager.cc:1905] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:56.163518 3711 ts_tablet_manager.cc:1918] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:56.163673 3711 log.cc:1196] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/fdbf55783c5346eea5b8da530b331215
I20260811 07:10:56.163987 3711 ts_tablet_manager.cc:1939] T fdbf55783c5346eea5b8da530b331215 P 8c8f36b356774ab293eb651c1ff471e1: Deleting consensus metadata
I20260811 07:10:56.164343 3710 ts_tablet_manager.cc:1918] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:56.164467 3710 log.cc:1196] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/fdbf55783c5346eea5b8da530b331215
I20260811 07:10:56.164763 3710 ts_tablet_manager.cc:1939] T fdbf55783c5346eea5b8da530b331215 P 9ffc0907be8c498598cbb7afbeca249d: Deleting consensus metadata
I20260811 07:10:56.165233 2910 catalog_manager.cc:4841] TS 9ffc0907be8c498598cbb7afbeca249d (127.3.236.130:46805): tablet fdbf55783c5346eea5b8da530b331215 (table test-table [id=df6ad6ac46124aa0a62192bf0163509a]) successfully deleted
I20260811 07:10:56.165293 2908 catalog_manager.cc:4841] TS 8c8f36b356774ab293eb651c1ff471e1 (127.3.236.131:45093): tablet fdbf55783c5346eea5b8da530b331215 (table test-table [id=df6ad6ac46124aa0a62192bf0163509a]) successfully deleted
W20260811 07:10:56.176092 2909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:44505 (error 108)
I20260811 07:10:56.176430 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 3272
I20260811 07:10:56.189642 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.189467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:44505 (local address 127.0.0.1:51096)
0811 07:10:56.189535 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.189595 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:44505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:10:56.189958 2909 catalog_manager.cc:4584] TS a12ef9c7320a49fc91fde9f922b4743f (127.3.236.129:44505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdbf55783c5346eea5b8da530b331215: Network error: Client connection negotiation failed: client connection to 127.3.236.129:44505: connect: Connection refused (error 111)
I20260811 07:10:56.199718 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 3470
I20260811 07:10:56.210711 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 2877
2026-08-11T07:10:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2245 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T07:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:56Z Disabled control of system clock
I20260811 07:10:56.244668 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36311
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:56.326124 3719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:56.326345 3719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:56.326367 3719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:56.327533 3719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:56.327577 3719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:56.327589 3719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:56.327602 3719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:56.327651 3719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:56.329028 3719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:36311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:56.329303 3719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:56.329617 3719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:56.334789 3734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.334826 3730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.334944 3719 server_base.cc:1034] running on GCE node
W20260811 07:10:56.334797 3729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.335381 3719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:56.335696 3719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:56.336840 3719 hybrid_clock.cc:648] HybridClock initialized: now 1786432256336830 us; error 48 us; skew 500 ppm
I20260811 07:10:56.338092 3719 webserver.cc:460] Webserver started at http://127.3.236.190:32941/ using document root <none> and password file <none>
I20260811 07:10:56.338367 3719 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:56.338415 3719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:56.338598 3719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:56.339751 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "7daf0ab760884668adc2b1a555b1c560"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "f85b0ccf15ebb139810a5a1fc537c4e4"
server_key_iv: "3454b3d9b01ea096764c2b9652442ad4"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.340132 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "7daf0ab760884668adc2b1a555b1c560"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "f85b0ccf15ebb139810a5a1fc537c4e4"
server_key_iv: "3454b3d9b01ea096764c2b9652442ad4"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.341476 3719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:56.342268 3743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.342459 3719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:10:56.342555 3719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "7daf0ab760884668adc2b1a555b1c560"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "f85b0ccf15ebb139810a5a1fc537c4e4"
server_key_iv: "3454b3d9b01ea096764c2b9652442ad4"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.342634 3719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:56.362353 3719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:56.362717 3719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:56.362883 3719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:56.367036 3719 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:36311
I20260811 07:10:56.367092 3825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:36311 every 8 connection(s)
I20260811 07:10:56.367470 3719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:56.368609 3826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.369067 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 3719
I20260811 07:10:56.369201 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:56.369398 4018 external_mini_cluster.cc:1428] Setting key d27126e53fc19b13ab207035ef1deece
I20260811 07:10:56.371851 3826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: Bootstrap starting.
I20260811 07:10:56.372617 3826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.372944 3826 log.cc:824] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.373723 3826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: No bootstrap required, opened a new log
I20260811 07:10:56.375725 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.369984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.3.236.190:36311)
0811 07:10:56.370233 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.370243 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.370279 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:10:56.370839 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.370848 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.370934 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.371088 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.371272 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.372684 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.373477 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.374080 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.374211 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.374861 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.374866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.374866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:56.375078 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.375125 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:56.375142 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.375212 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.375270 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.375355 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260811 07:10:56.375844 3826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } }
I20260811 07:10:56.376021 3826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.376068 3826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7daf0ab760884668adc2b1a555b1c560, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.376503 3826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } }
I20260811 07:10:56.376609 3826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:56.376657 3826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:56.376706 3826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.377238 3826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } }
I20260811 07:10:56.377362 3826 leader_election.cc:302] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7daf0ab760884668adc2b1a555b1c560; no voters:
I20260811 07:10:56.377553 3826 leader_election.cc:288] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:56.377620 3837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:56.377861 3837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [term 1 LEADER]: Becoming Leader. State: Replica: 7daf0ab760884668adc2b1a555b1c560, State: Running, Role: LEADER
I20260811 07:10:56.377899 3826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:56.377964 3837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } }
I20260811 07:10:56.378371 3840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7daf0ab760884668adc2b1a555b1c560. Latest consensus state: current_term: 1 leader_uuid: "7daf0ab760884668adc2b1a555b1c560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } } }
I20260811 07:10:56.378371 3838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7daf0ab760884668adc2b1a555b1c560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daf0ab760884668adc2b1a555b1c560" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 36311 } } }
I20260811 07:10:56.378485 3840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:56.378500 3838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560 [sys.catalog]: This master's current role is: LEADER
W20260811 07:10:56.379752 3861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:10:56.379819 3861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:10:56.379877 3846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:56.380070 3846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:56.381616 3846 catalog_manager.cc:1347] Generated new cluster ID: fa1cee6dbbd2413eb15d8218cd99ec56
I20260811 07:10:56.381672 3846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:56.392306 3846 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:56.392875 3846 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:56.403610 3846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: Generated new TSK 0
I20260811 07:10:56.403821 3846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:10:56.407171 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--builtin_ntp_servers=127.3.236.148:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:56.492208 3867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:56.492516 3867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:56.492556 3867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:56.493738 3867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:56.493781 3867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:56.493808 3867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:56.495159 3867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:56.495405 3867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:56.495725 3867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:56.501631 3872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.501624 3874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.501706 3876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.502053 3867 server_base.cc:1034] running on GCE node
I20260811 07:10:56.502280 3867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:56.502548 3867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:56.503703 3867 hybrid_clock.cc:648] HybridClock initialized: now 1786432256503688 us; error 39 us; skew 500 ppm
I20260811 07:10:56.505076 3867 webserver.cc:460] Webserver started at http://127.3.236.129:40833/ using document root <none> and password file <none>
I20260811 07:10:56.505311 3867 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:56.505378 3867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:56.505519 3867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:56.506600 3867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "5555ed001701432f8ef107e27449083c"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "bf20fc222d532017f9943e4a91aee419"
server_key_iv: "fdd7a9943053f46fe5933c84d3a762cf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.506970 3867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "5555ed001701432f8ef107e27449083c"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "bf20fc222d532017f9943e4a91aee419"
server_key_iv: "fdd7a9943053f46fe5933c84d3a762cf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.508335 3867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:56.509069 3883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.509243 3867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:56.509338 3867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "5555ed001701432f8ef107e27449083c"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "bf20fc222d532017f9943e4a91aee419"
server_key_iv: "fdd7a9943053f46fe5933c84d3a762cf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.509421 3867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:56.517722 3867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:56.518023 3867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:56.518174 3867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:56.518400 3867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:56.518723 3867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:56.518775 3867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.518811 3867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:56.518837 3867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.524731 3867 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:42389
I20260811 07:10:56.524791 4074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:42389 every 8 connection(s)
I20260811 07:10:56.525094 3867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:56.530071 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.525792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:44081 (local address 127.3.236.190:36311)
0811 07:10:56.525871 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.525873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.526092 (+ 219us) server_negotiation.cc:407] Connection header received
0811 07:10:56.526570 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.526572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.526587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.526644 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.527290 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.527785 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.528834 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.529021 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.529069 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.529835 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.529836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.529837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:56.529841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.529855 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:56.529864 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.529909 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.529984 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.530009 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:10:56.530303 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.525765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36311 (local address 127.3.236.129:44081)
0811 07:10:56.525970 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.526008 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.526481 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.526705 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.526708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.526717 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:56.527217 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.527221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.527868 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.527869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.528751 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.528754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.529091 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.529092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.529201 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.529705 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:56.529706 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:56.529734 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:56.529757 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:56.529917 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:56.529922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:56.529925 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.529976 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.530057 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260811 07:10:56.530870 4075 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36311
I20260811 07:10:56.530961 4075 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:56.531160 4075 heartbeater.cc:511] Master 127.3.236.190:36311 requested a full tablet report, sending...
I20260811 07:10:56.531685 3770 ts_manager.cc:194] Registered new tserver with Master: 5555ed001701432f8ef107e27449083c (127.3.236.129:42389)
I20260811 07:10:56.532359 3770 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.129:44081
I20260811 07:10:56.533229 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 3867
I20260811 07:10:56.533339 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:56.533535 4018 external_mini_cluster.cc:1428] Setting key 950ad60807790a3dd3be1460bb84ce33
I20260811 07:10:56.534399 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--builtin_ntp_servers=127.3.236.148:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:56.617594 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:56.617825 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:56.617864 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:56.619006 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:56.619067 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:56.619109 4080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:56.620539 4080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:56.620795 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:56.621083 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:56.626070 4092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.626137 4085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.626070 4086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.626415 4080 server_base.cc:1034] running on GCE node
I20260811 07:10:56.626657 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:56.626856 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:56.627992 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786432256627978 us; error 30 us; skew 500 ppm
I20260811 07:10:56.629346 4080 webserver.cc:460] Webserver started at http://127.3.236.130:33081/ using document root <none> and password file <none>
I20260811 07:10:56.629590 4080 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:56.629655 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:56.629835 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:56.631340 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "132f52ea8db74e5983500d56d0e8c55e"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "4eafa60444eed2a354fc74f7a64dd4dc"
server_key_iv: "7dc62e4fe24d51b35d24c6fbbc7f2d04"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.631716 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "132f52ea8db74e5983500d56d0e8c55e"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "4eafa60444eed2a354fc74f7a64dd4dc"
server_key_iv: "7dc62e4fe24d51b35d24c6fbbc7f2d04"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.633208 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:56.633970 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.634212 4080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:56.634356 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "132f52ea8db74e5983500d56d0e8c55e"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "4eafa60444eed2a354fc74f7a64dd4dc"
server_key_iv: "7dc62e4fe24d51b35d24c6fbbc7f2d04"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.634446 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:56.663595 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:56.663980 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:56.664146 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:56.664439 4080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:56.664806 4080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:56.664860 4080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.664896 4080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:56.664927 4080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.670765 4080 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:37111
I20260811 07:10:56.670849 4291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:37111 every 8 connection(s)
I20260811 07:10:56.671203 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:56.676344 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.672025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:44823 (local address 127.3.236.190:36311)
0811 07:10:56.672111 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.672113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.672387 (+ 274us) server_negotiation.cc:407] Connection header received
0811 07:10:56.672898 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.672899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.672909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.672969 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.673629 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.674120 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.675082 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.675260 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.675314 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.676101 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.676103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.676103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:56.676107 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.676121 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:56.676125 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.676168 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.676248 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.676285 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:10:56.676621 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.672019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36311 (local address 127.3.236.130:44823)
0811 07:10:56.672242 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.672309 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.672814 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.673034 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.673038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.673047 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:56.673562 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.673566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.674210 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.674211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.675006 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.675008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.675337 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.675338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.675448 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.675971 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:56.675971 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:56.676002 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:56.676024 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:56.676178 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:56.676183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:56.676185 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.676239 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.676389 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260811 07:10:56.677096 4293 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36311
I20260811 07:10:56.677187 4293 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:56.677340 4293 heartbeater.cc:511] Master 127.3.236.190:36311 requested a full tablet report, sending...
I20260811 07:10:56.677687 3770 ts_manager.cc:194] Registered new tserver with Master: 132f52ea8db74e5983500d56d0e8c55e (127.3.236.130:37111)
I20260811 07:10:56.678040 3770 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.130:44823
I20260811 07:10:56.680577 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4080
I20260811 07:10:56.680684 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:10:56.680887 4018 external_mini_cluster.cc:1428] Setting key 64858c2e6ec4f8897ed65edd8c67fef6
I20260811 07:10:56.681743 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--builtin_ntp_servers=127.3.236.148:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:56.766920 4298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:56.767168 4298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:56.767205 4298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:56.768345 4298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:56.768409 4298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:56.768448 4298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:56.769749 4298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:56.770028 4298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:56.770331 4298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:56.775489 4304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.775588 4310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:56.775679 4305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:56.775782 4298 server_base.cc:1034] running on GCE node
I20260811 07:10:56.776046 4298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:56.776378 4298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:56.777531 4298 hybrid_clock.cc:648] HybridClock initialized: now 1786432256777518 us; error 32 us; skew 500 ppm
I20260811 07:10:56.779033 4298 webserver.cc:460] Webserver started at http://127.3.236.131:35131/ using document root <none> and password file <none>
I20260811 07:10:56.779311 4298 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:56.779361 4298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:56.779531 4298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:56.780807 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "9b52aaf7c0424c1c9580f3f86ea1f717"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "31975a3b7fa26773e5899331f14cab81"
server_key_iv: "0e2d89b8a41359617901b8de1cc06cdf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.781199 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "9b52aaf7c0424c1c9580f3f86ea1f717"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "31975a3b7fa26773e5899331f14cab81"
server_key_iv: "0e2d89b8a41359617901b8de1cc06cdf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.782626 4298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:10:56.783418 4320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.783605 4298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:10:56.783720 4298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "9b52aaf7c0424c1c9580f3f86ea1f717"
format_stamp: "Formatted at 2026-08-11 07:10:56 on dist-test-slave-2d19"
server_key: "31975a3b7fa26773e5899331f14cab81"
server_key_iv: "0e2d89b8a41359617901b8de1cc06cdf"
server_key_version: "encryptionkey@0"
I20260811 07:10:56.783799 4298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:56.809888 4298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:56.810288 4298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:56.810473 4298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:56.810760 4298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:56.811208 4298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:56.811270 4298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.811318 4298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:56.811350 4298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:56.817912 4298 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:42627
I20260811 07:10:56.817966 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:42627 every 8 connection(s)
I20260811 07:10:56.818382 4298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:10:56.824615 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.819251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38761 (local address 127.3.236.190:36311)
0811 07:10:56.819310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.819312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.819633 (+ 321us) server_negotiation.cc:407] Connection header received
0811 07:10:56.820491 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.820492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.820506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.820589 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.821417 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.822111 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.823269 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.823457 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.823530 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.824399 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.824400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.824401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:56.824405 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.824419 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:56.824422 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.824466 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.824535 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.824555 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:10:56.824879 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.819254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:36311 (local address 127.3.236.131:38761)
0811 07:10:56.819480 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.819529 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.820314 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.820678 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.820684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.820696 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:56.821319 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.821325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.822197 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.822198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.823181 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.823184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.823555 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.823556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.823677 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.824206 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:56.824206 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:56.824237 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:56.824313 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:56.824475 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:56.824481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:56.824483 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.824539 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.824624 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260811 07:10:56.825368 4501 heartbeater.cc:347] Connected to a master server at 127.3.236.190:36311
I20260811 07:10:56.825491 4501 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:56.825685 4501 heartbeater.cc:511] Master 127.3.236.190:36311 requested a full tablet report, sending...
I20260811 07:10:56.826079 3770 ts_manager.cc:194] Registered new tserver with Master: 9b52aaf7c0424c1c9580f3f86ea1f717 (127.3.236.131:42627)
I20260811 07:10:56.826447 3770 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.131:38761
I20260811 07:10:56.827663 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4298
I20260811 07:10:56.827816 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:10:56.828025 4018 external_mini_cluster.cc:1428] Setting key 1bbd701155884d59cfa3b91bdb6681ab
I20260811 07:10:56.828974 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:56.833626 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.830484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.3.236.190:36311)
0811 07:10:56.830554 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.830556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.830660 (+ 104us) server_negotiation.cc:407] Connection header received
0811 07:10:56.830701 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.830702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.830711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.830769 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:56.830925 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.831397 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.832180 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.832619 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.832675 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.833317 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:56.833319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:56.833319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:56.833323 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:56.833336 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:56.833340 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:56.833428 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.833560 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.833571 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:10:56.835186 3770 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:56.836866 3770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:56.843030 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.838955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:42389 (local address 127.0.0.1:52930)
0811 07:10:56.839247 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.839257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.839299 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.839593 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.839596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.839598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.839647 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.839648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.840771 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.840772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.841902 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.841905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.842741 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.842742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.842817 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.842830 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.842896 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.842952 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260811 07:10:56.843044 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.839061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52930 (local address 127.3.236.129:42389)
0811 07:10:56.839408 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.839413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.839428 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:10:56.839460 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.839462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.839489 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.839580 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.839743 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.840690 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.842000 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.842657 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.842741 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.842835 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.842923 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.842960 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":97,"threads_started":1}
I20260811 07:10:56.843456 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.838942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:42627 (local address 127.0.0.1:58322)
0811 07:10:56.839156 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.839198 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.839270 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.839872 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.839872 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.839875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.839909 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.839911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.841228 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.841229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.842339 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.842341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.843225 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.843226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.843293 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.843297 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.843329 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.843368 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":28,"thread_start_us":63,"threads_started":1}
I20260811 07:10:56.843648 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.839381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.3.236.130:37111)
0811 07:10:56.839599 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.839603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.839612 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:56.839653 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.839655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.839669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.839747 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.839950 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.841116 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.842395 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.843116 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.843192 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.843285 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.843501 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.843544 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
I20260811 07:10:56.843897 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.839034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.3.236.131:42627)
0811 07:10:56.839714 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.839718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.839728 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:10:56.839762 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.839763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.839777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.839984 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.840050 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.841146 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.842427 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.843116 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.843671 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.843776 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.843801 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.843839 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":79,"threads_started":1}
I20260811 07:10:56.844038 3963 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe51098f2914528bb454bf13a63f14d (DEFAULT_TABLE table=test-table [id=139d4746138e4b1ab51974a6b9493969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.844471 3963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe51098f2914528bb454bf13a63f14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.844646 4395 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe51098f2914528bb454bf13a63f14d (DEFAULT_TABLE table=test-table [id=139d4746138e4b1ab51974a6b9493969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.844990 4395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe51098f2914528bb454bf13a63f14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.848124 4526 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: Bootstrap starting.
I20260811 07:10:56.848481 4527 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Bootstrap starting.
I20260811 07:10:56.848519 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.839156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:37111 (local address 127.0.0.1:39964)
0811 07:10:56.839298 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.839310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.839364 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.839820 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.839821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.839823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.839871 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.839872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.841223 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.841224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.842152 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.842155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.843317 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.843318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.843368 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.843371 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.843417 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.843446 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":28,"thread_start_us":87,"threads_started":1}
I20260811 07:10:56.848919 4526 tablet_bootstrap.cc:654] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.849263 4526 log.cc:824] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.849298 4527 tablet_bootstrap.cc:654] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.849315 4179 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe51098f2914528bb454bf13a63f14d (DEFAULT_TABLE table=test-table [id=139d4746138e4b1ab51974a6b9493969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:56.849629 4527 log.cc:824] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.849687 4179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe51098f2914528bb454bf13a63f14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:56.850090 4526 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: No bootstrap required, opened a new log
I20260811 07:10:56.850167 4526 ts_tablet_manager.cc:1397] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:56.850369 4527 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: No bootstrap required, opened a new log
I20260811 07:10:56.850449 4527 ts_tablet_manager.cc:1397] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:10:56.852366 4527 raft_consensus.cc:348] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.852551 4527 raft_consensus.cc:374] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.852598 4527 raft_consensus.cc:729] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b52aaf7c0424c1c9580f3f86ea1f717, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.852736 4527 consensus_queue.cc:260] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.853041 4527 ts_tablet_manager.cc:1428] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:56.853046 4501 heartbeater.cc:503] Master 127.3.236.190:36311 was elected leader, sending a full tablet report...
I20260811 07:10:56.852958 4526 raft_consensus.cc:348] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.853117 4526 raft_consensus.cc:374] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.853161 4526 raft_consensus.cc:729] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5555ed001701432f8ef107e27449083c, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.853286 4526 consensus_queue.cc:260] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.853578 4526 ts_tablet_manager.cc:1428] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 07:10:56.853737 4075 heartbeater.cc:503] Master 127.3.236.190:36311 was elected leader, sending a full tablet report...
I20260811 07:10:56.853621 4535 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Bootstrap starting.
I20260811 07:10:56.854480 4535 tablet_bootstrap.cc:654] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:56.854837 4535 log.cc:824] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:56.855501 4535 tablet_bootstrap.cc:492] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: No bootstrap required, opened a new log
I20260811 07:10:56.855558 4535 ts_tablet_manager.cc:1397] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:10:56.857678 4535 raft_consensus.cc:348] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.857810 4535 raft_consensus.cc:374] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:56.857831 4535 raft_consensus.cc:729] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132f52ea8db74e5983500d56d0e8c55e, State: Initialized, Role: FOLLOWER
I20260811 07:10:56.857969 4535 consensus_queue.cc:260] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.858219 4535 ts_tablet_manager.cc:1428] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:56.858278 4293 heartbeater.cc:503] Master 127.3.236.190:36311 was elected leader, sending a full tablet report...
I20260811 07:10:56.901191 4531 raft_consensus.cc:482] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:56.901342 4531 raft_consensus.cc:504] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.901743 4531 leader_election.cc:288] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5555ed001701432f8ef107e27449083c (127.3.236.129:42389), 132f52ea8db74e5983500d56d0e8c55e (127.3.236.130:37111)
I20260811 07:10:56.905632 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.902041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:49995 (local address 127.3.236.129:42389)
0811 07:10:56.902094 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.902096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.902109 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:10:56.902145 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.902146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.902155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.902227 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.902402 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.903164 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.904712 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.905268 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.905346 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.905353 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.905518 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.905561 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 07:10:56.905632 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.902055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:57509 (local address 127.3.236.130:37111)
0811 07:10:56.902099 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.902102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.902215 (+ 113us) server_negotiation.cc:407] Connection header received
0811 07:10:56.902289 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.902289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.902301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.902346 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:56.902503 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.903179 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.904711 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.905268 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.905345 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.905353 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.905524 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.905561 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:10:56.905637 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.901916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:42389 (local address 127.3.236.131:49995)
0811 07:10:56.901983 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.901996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.902059 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.902236 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.902237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.902239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.902314 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.902316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.903256 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.903257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.904620 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.904624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.905356 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.905357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.905436 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.905442 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.905497 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.905548 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":53}
I20260811 07:10:56.905650 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.901923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:37111 (local address 127.3.236.131:57509)
0811 07:10:56.902144 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.902158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:56.902232 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:56.902365 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:56.902366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:56.902368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:56.902428 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.902434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.903258 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.903259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.904620 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:56.904624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.905373 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.905373 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:56.905436 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.905442 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:10:56.905497 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:10:56.905571 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":88,"thread_start_us":111,"threads_started":1}
I20260811 07:10:56.906086 4211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe51098f2914528bb454bf13a63f14d" candidate_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132f52ea8db74e5983500d56d0e8c55e" is_pre_election: true
I20260811 07:10:56.906279 4211 raft_consensus.cc:2454] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b52aaf7c0424c1c9580f3f86ea1f717 in term 0.
I20260811 07:10:56.906319 3992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe51098f2914528bb454bf13a63f14d" candidate_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5555ed001701432f8ef107e27449083c" is_pre_election: true
I20260811 07:10:56.906481 3992 raft_consensus.cc:2454] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b52aaf7c0424c1c9580f3f86ea1f717 in term 0.
I20260811 07:10:56.906548 4322 leader_election.cc:302] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 132f52ea8db74e5983500d56d0e8c55e, 9b52aaf7c0424c1c9580f3f86ea1f717; no voters:
I20260811 07:10:56.906710 4531 raft_consensus.cc:2781] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:56.906786 4531 raft_consensus.cc:482] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:56.906803 4531 raft_consensus.cc:3037] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.907507 4531 raft_consensus.cc:504] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.907651 4531 leader_election.cc:288] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [CANDIDATE]: Term 1 election: Requested vote from peers 5555ed001701432f8ef107e27449083c (127.3.236.129:42389), 132f52ea8db74e5983500d56d0e8c55e (127.3.236.130:37111)
I20260811 07:10:56.907796 3992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe51098f2914528bb454bf13a63f14d" candidate_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5555ed001701432f8ef107e27449083c"
I20260811 07:10:56.907823 4211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe51098f2914528bb454bf13a63f14d" candidate_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132f52ea8db74e5983500d56d0e8c55e"
I20260811 07:10:56.907891 3992 raft_consensus.cc:3037] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.907897 4211 raft_consensus.cc:3037] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:56.908703 3992 raft_consensus.cc:2454] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b52aaf7c0424c1c9580f3f86ea1f717 in term 1.
I20260811 07:10:56.908748 4211 raft_consensus.cc:2454] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b52aaf7c0424c1c9580f3f86ea1f717 in term 1.
I20260811 07:10:56.908911 4322 leader_election.cc:302] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 132f52ea8db74e5983500d56d0e8c55e, 9b52aaf7c0424c1c9580f3f86ea1f717; no voters:
I20260811 07:10:56.908998 4531 raft_consensus.cc:2781] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:56.909148 4531 raft_consensus.cc:686] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 1 LEADER]: Becoming Leader. State: Replica: 9b52aaf7c0424c1c9580f3f86ea1f717, State: Running, Role: LEADER
I20260811 07:10:56.909246 4531 consensus_queue.cc:237] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } }
I20260811 07:10:56.910140 3768 catalog_manager.cc:5495] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b52aaf7c0424c1c9580f3f86ea1f717 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b52aaf7c0424c1c9580f3f86ea1f717" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 42627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260811 07:10:56.917886 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.915451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.3.236.131:42627)
0811 07:10:56.915512 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:56.915514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:56.915523 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:56.915556 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:56.915557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:56.915578 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:56.915659 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:56.915803 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.916316 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.917078 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:56.917282 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:56.917356 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:56.917527 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:56.917620 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:56.917727 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:56.917776 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:10:56.917785 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:10:56.919620 3992 raft_consensus.cc:1261] T 0fe51098f2914528bb454bf13a63f14d P 5555ed001701432f8ef107e27449083c [term 1 FOLLOWER]: Refusing update from remote peer 9b52aaf7c0424c1c9580f3f86ea1f717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:56.919618 4211 raft_consensus.cc:1261] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 1 FOLLOWER]: Refusing update from remote peer 9b52aaf7c0424c1c9580f3f86ea1f717: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:56.919978 4531 consensus_queue.cc:1035] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5555ed001701432f8ef107e27449083c" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:56.920147 4540 consensus_queue.cc:1035] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [LEADER]: Connected to new peer: Peer: permanent_uuid: "132f52ea8db74e5983500d56d0e8c55e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:10:56.921729 4294 tablet.cc:2369] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:10:56.922709 4546 mvcc.cc:205] Tried to move back new op lower bound from 7317226524340322304 to 7317226524301049856. Current Snapshot: MvccSnapshot[applied={T|T < 7317226524340322304}]
I20260811 07:10:56.923463 4547 mvcc.cc:205] Tried to move back new op lower bound from 7317226524340322304 to 7317226524301049856. Current Snapshot: MvccSnapshot[applied={T|T < 7317226524340322304}]
I20260811 07:10:56.923552 4548 mvcc.cc:205] Tried to move back new op lower bound from 7317226524340322304 to 7317226524301049856. Current Snapshot: MvccSnapshot[applied={T|T < 7317226524340322304}]
I20260811 07:10:56.925789 3770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:56.925891 3770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:56.926712 3770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7daf0ab760884668adc2b1a555b1c560: Sending DeleteTablet for 3 replicas of tablet 0fe51098f2914528bb454bf13a63f14d
I20260811 07:10:56.927073 4395 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe51098f2914528bb454bf13a63f14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='slave'} at 127.0.0.1:58322
I20260811 07:10:56.927321 3963 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe51098f2914528bb454bf13a63f14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='slave'} at 127.0.0.1:52930
I20260811 07:10:56.927347 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 3867
I20260811 07:10:56.927379 4563 tablet_replica.cc:330] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: stopping tablet replica
I20260811 07:10:56.958575 4563 raft_consensus.cc:2229] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:10:56.958777 4563 raft_consensus.cc:2258] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:56.958748 4179 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fe51098f2914528bb454bf13a63f14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:56 UTC) from {username='slave'} at 127.0.0.1:39964
I20260811 07:10:56.959142 4564 tablet_replica.cc:330] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: stopping tablet replica
I20260811 07:10:56.959326 4564 raft_consensus.cc:2229] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:56.959513 4564 raft_consensus.cc:2258] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:56.959591 4563 ts_tablet_manager.cc:1905] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:56.960124 4564 ts_tablet_manager.cc:1905] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:56.961691 4563 ts_tablet_manager.cc:1918] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:56.961782 4563 log.cc:1196] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/0fe51098f2914528bb454bf13a63f14d
I20260811 07:10:56.962069 4563 ts_tablet_manager.cc:1939] T 0fe51098f2914528bb454bf13a63f14d P 9b52aaf7c0424c1c9580f3f86ea1f717: Deleting consensus metadata
I20260811 07:10:56.962477 4564 ts_tablet_manager.cc:1918] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:56.962564 3745 catalog_manager.cc:4841] TS 9b52aaf7c0424c1c9580f3f86ea1f717 (127.3.236.131:42627): tablet 0fe51098f2914528bb454bf13a63f14d (table test-table [id=139d4746138e4b1ab51974a6b9493969]) successfully deleted
I20260811 07:10:56.962630 4564 log.cc:1196] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/0fe51098f2914528bb454bf13a63f14d
I20260811 07:10:56.962965 4564 ts_tablet_manager.cc:1939] T 0fe51098f2914528bb454bf13a63f14d P 132f52ea8db74e5983500d56d0e8c55e: Deleting consensus metadata
I20260811 07:10:56.963449 3745 catalog_manager.cc:4841] TS 132f52ea8db74e5983500d56d0e8c55e (127.3.236.130:37111): tablet 0fe51098f2914528bb454bf13a63f14d (table test-table [id=139d4746138e4b1ab51974a6b9493969]) successfully deleted
W20260811 07:10:56.964334 3754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:42389 (error 108)
I20260811 07:10:56.964689 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4080
I20260811 07:10:56.964862 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:56.964695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:42389 (local address 127.0.0.1:52932)
0811 07:10:56.964770 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:56.964819 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:10:56.969022 3754 catalog_manager.cc:4584] TS 5555ed001701432f8ef107e27449083c (127.3.236.129:42389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fe51098f2914528bb454bf13a63f14d: Network error: Client connection negotiation failed: client connection to 127.3.236.129:42389: connect: Connection refused (error 111)
I20260811 07:10:56.984807 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4298
I20260811 07:10:56.992527 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 3719
2026-08-11T07:10:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (790 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:10:57 dist-test-slave-2d19 krb5kdc[4571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:10:57 dist-test-slave-2d19 krb5kdc[4571](info): set up 2 sockets
Aug 11 07:10:57 dist-test-slave-2d19 krb5kdc[4571](info): commencing operation
krb5kdc: starting...
W20260811 07:10:58.294019 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:10:58.421114 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35583
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:58.504385 4590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:58.504616 4590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:58.504655 4590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:58.505800 4590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:58.505864 4590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:58.505888 4590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:58.505911 4590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:58.505937 4590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:58.507402 4590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:58.507668 4590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:58.507902 4590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:10:58.513041 4590 server_base.cc:1034] running on GCE node
W20260811 07:10:58.512985 4596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.512997 4595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.512987 4598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:58.513497 4590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:58.513784 4590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:58.514971 4590 hybrid_clock.cc:648] HybridClock initialized: now 1786432258514954 us; error 32 us; skew 500 ppm
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:58.518085 4590 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:10:58.518518 4590 webserver.cc:460] Webserver started at http://127.3.236.190:33589/ using document root <none> and password file <none>
I20260811 07:10:58.518770 4590 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:58.518839 4590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:58.518978 4590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:58.520047 4590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "830343183d6a4b3cbf383d400e7f7775"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "70189dcb98a9f3218ca506664bb78d21"
server_key_iv: "6bcf1db3d5a92dfda401842bff69a78a"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.520444 4590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "830343183d6a4b3cbf383d400e7f7775"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "70189dcb98a9f3218ca506664bb78d21"
server_key_iv: "6bcf1db3d5a92dfda401842bff69a78a"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.521782 4590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:58.522535 4615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.522701 4590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:10:58.522805 4590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "830343183d6a4b3cbf383d400e7f7775"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "70189dcb98a9f3218ca506664bb78d21"
server_key_iv: "6bcf1db3d5a92dfda401842bff69a78a"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.522878 4590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:58.542866 4590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:58.545500 4590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:58.545660 4590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:58.549790 4590 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35583
I20260811 07:10:58.549827 4694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35583 every 8 connection(s)
I20260811 07:10:58.550196 4590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:58.551110 4695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:58.553555 4695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: Bootstrap starting.
I20260811 07:10:58.554163 4695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:58.554450 4695 log.cc:824] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:58.555156 4695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: No bootstrap required, opened a new log
I20260811 07:10:58.556790 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4590
I20260811 07:10:58.556919 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:58.556891 4695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } }
I20260811 07:10:58.557092 4695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:58.557144 4695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830343183d6a4b3cbf383d400e7f7775, State: Initialized, Role: FOLLOWER
I20260811 07:10:58.557161 4018 external_mini_cluster.cc:1428] Setting key 5a32b7e1b283d90ba68f2c4c619da70b
I20260811 07:10:58.557307 4695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } }
I20260811 07:10:58.557413 4695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:58.557471 4695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:58.557529 4695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:58.558630 4695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } }
I20260811 07:10:58.558805 4695 leader_election.cc:302] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830343183d6a4b3cbf383d400e7f7775; no voters:
I20260811 07:10:58.558979 4695 leader_election.cc:288] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:58.559027 4705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:58.559218 4705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [term 1 LEADER]: Becoming Leader. State: Replica: 830343183d6a4b3cbf383d400e7f7775, State: Running, Role: LEADER
I20260811 07:10:58.559343 4695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:58.559367 4705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } }
I20260811 07:10:58.559836 4707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "830343183d6a4b3cbf383d400e7f7775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } } }
I20260811 07:10:58.559927 4707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:58.560180 4706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830343183d6a4b3cbf383d400e7f7775. Latest consensus state: current_term: 1 leader_uuid: "830343183d6a4b3cbf383d400e7f7775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830343183d6a4b3cbf383d400e7f7775" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35583 } } }
I20260811 07:10:58.560609 4706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:58.560773 4717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:58.561240 4717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:58.562722 4717 catalog_manager.cc:1347] Generated new cluster ID: c35566b2164146b09020012558311130
I20260811 07:10:58.562781 4717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:58.568779 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:58.557670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.3.236.190:35583)
0811 07:10:58.557844 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:58.557853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:58.557887 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:10:58.558005 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:58.558010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:58.558052 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:58.558160 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:58.559179 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.560676 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.561784 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.562379 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.562490 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:58.564963 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:58.564967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:58.564967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:58.564980 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:58.566657 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.567045 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.567048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.567049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.567668 (+ 619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.567894 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.567895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.567895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.568138 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:58.568356 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:58.568433 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 07:10:58.568595 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260811 07:10:58.569937 4717 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:58.570416 4717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:58.574756 4717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: Generated new TSK 0
I20260811 07:10:58.574927 4717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
Entry for principal kudu/127.3.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab.
I20260811 07:10:58.607059 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:58.712006 4735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:58.712206 4735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:58.712229 4735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:58.713464 4735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:58.713510 4735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:58.713537 4735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:58.715342 4735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.129.keytab
--principal=kudu/127.3.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:58.715840 4735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:58.716167 4735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:58.721556 4746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.721556 4742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.721570 4743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:58.721762 4735 server_base.cc:1034] running on GCE node
I20260811 07:10:58.722052 4735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:58.722311 4735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:58.723474 4735 hybrid_clock.cc:648] HybridClock initialized: now 1786432258723461 us; error 32 us; skew 500 ppm
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:58.726547 4735 init.cc:377] Logged in from keytab as kudu/127.3.236.129@KRBTEST.COM (short username kudu)
I20260811 07:10:58.727066 4735 webserver.cc:460] Webserver started at http://127.3.236.129:39959/ using document root <none> and password file <none>
I20260811 07:10:58.727272 4735 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:58.727313 4735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:58.727401 4735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:58.728485 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "90caa4cc0596564e5c1000d92372f1d6"
server_key_iv: "e7953e9844037e6e72762fd458e6fc10"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.728786 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "90caa4cc0596564e5c1000d92372f1d6"
server_key_iv: "e7953e9844037e6e72762fd458e6fc10"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.730417 4735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:58.731132 4758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.731338 4735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:58.731423 4735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "90caa4cc0596564e5c1000d92372f1d6"
server_key_iv: "e7953e9844037e6e72762fd458e6fc10"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.731480 4735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:58.738670 4735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:58.741034 4735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:58.741147 4735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:58.741323 4735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:58.741655 4735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:58.741690 4735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.741714 4735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:58.741761 4735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.747901 4735 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:40849
I20260811 07:10:58.748018 4936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:40849 every 8 connection(s)
I20260811 07:10:58.748382 4735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:58.753114 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4735
I20260811 07:10:58.753257 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:58.753492 4018 external_mini_cluster.cc:1428] Setting key bae08ee62fbc7c64763a2af30958dbfc
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.129@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:58.758682 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:58.749325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:46135 (local address 127.3.236.190:35583)
0811 07:10:58.749399 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:58.749401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:58.749649 (+ 248us) server_negotiation.cc:407] Connection header received
0811 07:10:58.750146 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:58.750147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:58.750161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:58.750232 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:58.751193 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.751778 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.752887 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.753108 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.753169 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:58.755467 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:58.755469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:58.755469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:58.755477 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:58.757207 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.757639 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.757640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.757640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.757694 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.757994 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.757995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.757995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.758121 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:58.758203 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:58.758499 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 07:10:58.758592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:10:58.758895 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:58.749299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35583 (local address 127.3.236.129:46135)
0811 07:10:58.749514 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:58.749553 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:58.750051 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:58.750299 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:58.750303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:58.750883 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:58.751093 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:58.751098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.751864 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.751865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:58.752794 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:58.752797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.753205 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.753206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:58.753377 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:58.753976 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:58.753985 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:58.755368 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:58.757311 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:58.757318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:58.757320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:58.757569 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:58.757803 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:58.757804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:58.757804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:58.757921 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:58.758195 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:58.758202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:58.758348 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 07:10:58.758456 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 07:10:58.758624 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260811 07:10:58.759581 4937 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35583
I20260811 07:10:58.759701 4937 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:58.759900 4937 heartbeater.cc:511] Master 127.3.236.190:35583 requested a full tablet report, sending...
I20260811 07:10:58.760581 4641 ts_manager.cc:194] Registered new tserver with Master: 3eea1dfd3bbd4c98a4bf8648dd10e0a0 (127.3.236.129:40849)
I20260811 07:10:58.761622 4641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.129@KRBTEST.COM'} at 127.3.236.129:46135
WARNING: no policy specified for kudu/127.3.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
Entry for principal kudu/127.3.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab.
I20260811 07:10:58.783015 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:58.868165 4945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:58.868433 4945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:58.868471 4945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:58.869612 4945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:58.869681 4945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:58.869727 4945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:10:58.871203 4945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.130.keytab
--principal=kudu/127.3.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:58.871480 4945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:58.871722 4945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:58.876765 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.876780 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:58.876780 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:58.877092 4945 server_base.cc:1034] running on GCE node
I20260811 07:10:58.877303 4945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:58.877508 4945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:58.878649 4945 hybrid_clock.cc:648] HybridClock initialized: now 1786432258878638 us; error 27 us; skew 500 ppm
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:58.881318 4945 init.cc:377] Logged in from keytab as kudu/127.3.236.130@KRBTEST.COM (short username kudu)
I20260811 07:10:58.881772 4945 webserver.cc:460] Webserver started at http://127.3.236.130:35819/ using document root <none> and password file <none>
I20260811 07:10:58.882025 4945 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:58.882095 4945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:58.882247 4945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:58.883301 4945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "481644ff2caa42c398d97e1c7e09154e"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "11f6623b6a79a5f3d9ec54917c57ed53"
server_key_iv: "bb7ab26e64bf507d576342381d6cd071"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.883675 4945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "481644ff2caa42c398d97e1c7e09154e"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "11f6623b6a79a5f3d9ec54917c57ed53"
server_key_iv: "bb7ab26e64bf507d576342381d6cd071"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.885115 4945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:10:58.885881 4971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.886130 4945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:58.886219 4945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "481644ff2caa42c398d97e1c7e09154e"
format_stamp: "Formatted at 2026-08-11 07:10:58 on dist-test-slave-2d19"
server_key: "11f6623b6a79a5f3d9ec54917c57ed53"
server_key_iv: "bb7ab26e64bf507d576342381d6cd071"
server_key_version: "encryptionkey@0"
I20260811 07:10:58.886273 4945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:58.906373 4945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:58.908833 4945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:58.908950 4945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:58.909137 4945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:58.909549 4945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:58.909591 4945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.909614 4945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:58.909660 4945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:58.915675 4945 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:33687
I20260811 07:10:58.915752 5151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:33687 every 8 connection(s)
I20260811 07:10:58.916083 4945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:10:58.918715 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 4945
I20260811 07:10:58.918872 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:10:58.919080 4018 external_mini_cluster.cc:1428] Setting key 3bdc481140538fd9f3c67ebb567dc779
Aug 11 07:10:58 dist-test-slave-2d19 krb5kdc[4571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432258, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.130@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:58.926596 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:58.916864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:47051 (local address 127.3.236.190:35583)
0811 07:10:58.916937 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:58.916939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:58.917158 (+ 219us) server_negotiation.cc:407] Connection header received
0811 07:10:58.917659 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:58.917660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:58.917671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:58.917741 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:58.918778 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.919480 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.920659 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.920975 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.921059 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:58.923442 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:58.923444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:58.923444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:58.923452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:58.925002 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.925518 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.925519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.925520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.925582 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:58.925860 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:58.925861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:58.925862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:58.925977 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:58.926086 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:58.926377 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 07:10:58.926506 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:10:58.926755 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:58.916848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35583 (local address 127.3.236.130:47051)
0811 07:10:58.917036 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:58.917074 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:58.917543 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:58.917841 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:58.917846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:58.918459 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:58.918674 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:58.918680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.919569 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.919570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:58.920574 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:58.920577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:58.921064 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:58.921065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:58.921166 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:58.921654 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:58.921662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:58.923321 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:58.925095 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:58.925103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:58.925105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:58.925419 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:58.925657 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:58.925658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:58.925658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:58.925805 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:58.926090 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:58.926099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:58.926238 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 07:10:58.926343 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 07:10:58.926520 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260811 07:10:58.927418 5152 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35583
I20260811 07:10:58.927532 5152 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:58.927719 5152 heartbeater.cc:511] Master 127.3.236.190:35583 requested a full tablet report, sending...
I20260811 07:10:58.928169 4641 ts_manager.cc:194] Registered new tserver with Master: 481644ff2caa42c398d97e1c7e09154e (127.3.236.130:33687)
I20260811 07:10:58.928752 4641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.130@KRBTEST.COM'} at 127.3.236.130:47051
WARNING: no policy specified for kudu/127.3.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
Entry for principal kudu/127.3.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab.
I20260811 07:10:58.950889 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:10:59.035401 5161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:59.035600 5161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:59.035625 5161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:59.036787 5161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:59.036828 5161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:59.036854 5161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:10:59.038307 5161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.131.keytab
--principal=kudu/127.3.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:59.038551 5161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:59.038801 5161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:59.043862 5171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:59.043933 5169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:59.044070 5161 server_base.cc:1034] running on GCE node
W20260811 07:10:59.043870 5168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:59.044394 5161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:59.044660 5161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:59.045838 5161 hybrid_clock.cc:648] HybridClock initialized: now 1786432259045825 us; error 39 us; skew 500 ppm
Aug 11 07:10:59 dist-test-slave-2d19 krb5kdc[4571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:10:59.048552 5161 init.cc:377] Logged in from keytab as kudu/127.3.236.131@KRBTEST.COM (short username kudu)
I20260811 07:10:59.049048 5161 webserver.cc:460] Webserver started at http://127.3.236.131:43043/ using document root <none> and password file <none>
I20260811 07:10:59.049295 5161 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:59.049366 5161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:59.049537 5161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:59.050631 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "5f39f3b396684136a1a24a9ebf3480b7"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "70cd71cfdeb1ef63c7130c41cee4c34f"
server_key_iv: "b21abbfe38b6abbf222978104abc4616"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.050994 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "5f39f3b396684136a1a24a9ebf3480b7"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "70cd71cfdeb1ef63c7130c41cee4c34f"
server_key_iv: "b21abbfe38b6abbf222978104abc4616"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.052489 5161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:10:59.053287 5181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.053468 5161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:10:59.053577 5161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "5f39f3b396684136a1a24a9ebf3480b7"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "70cd71cfdeb1ef63c7130c41cee4c34f"
server_key_iv: "b21abbfe38b6abbf222978104abc4616"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.053653 5161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:59.098170 5161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:59.100224 5161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:59.100421 5161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:59.100677 5161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:59.101004 5161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:59.101040 5161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.101063 5161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:59.101110 5161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.106967 5161 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:35809
I20260811 07:10:59.107064 5390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:35809 every 8 connection(s)
I20260811 07:10:59.107379 5161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:10:59 dist-test-slave-2d19 krb5kdc[4571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432259, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.131@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:10:59.116768 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.108173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:46743 (local address 127.3.236.190:35583)
0811 07:10:59.108240 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.108242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.108466 (+ 224us) server_negotiation.cc:407] Connection header received
0811 07:10:59.109020 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.109021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.109035 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.109076 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:59.110148 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.110661 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.111691 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.111908 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.111985 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.114138 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:59.114139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:59.114140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:59.114148 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:59.115327 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:59.115755 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:59.115756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:59.115757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:59.115818 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:59.116089 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:59.116090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:59.116091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:59.116206 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:59.116296 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.116587 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.116677 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:10:59.116950 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.108132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:35583 (local address 127.3.236.131:46743)
0811 07:10:59.108335 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.108383 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.108901 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.109159 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.109163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.109824 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:59.110034 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.110040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.110774 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.110775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.111584 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.111586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.112006 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.112007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.112138 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.112647 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:10:59.112655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:59.114036 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:59.115429 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:59.115437 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:59.115439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:59.115665 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:59.115893 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:10:59.115894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:10:59.115894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:10:59.116008 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:10:59.116295 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:59.116304 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:59.116440 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.116560 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.116717 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":132,"threads_started":1}
I20260811 07:10:59.117194 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5161
I20260811 07:10:59.117318 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:10:59.117429 5391 heartbeater.cc:347] Connected to a master server at 127.3.236.190:35583
I20260811 07:10:59.117518 4018 external_mini_cluster.cc:1428] Setting key 5ae75be5f49bc549ed39266be4cee965
I20260811 07:10:59.117545 5391 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:59.117743 5391 heartbeater.cc:511] Master 127.3.236.190:35583 requested a full tablet report, sending...
I20260811 07:10:59.118144 4641 ts_manager.cc:194] Registered new tserver with Master: 5f39f3b396684136a1a24a9ebf3480b7 (127.3.236.131:35809)
I20260811 07:10:59.118443 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:10:59.118621 4641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.236.131@KRBTEST.COM'} at 127.3.236.131:46743
I20260811 07:10:59.126075 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.119897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.3.236.190:35583)
0811 07:10:59.119989 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.119991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.120127 (+ 136us) server_negotiation.cc:407] Connection header received
0811 07:10:59.120197 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.120198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.120208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.120289 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:59.120822 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.121288 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.122042 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.122245 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.122317 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.123540 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:59.123542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:59.123542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:10:59.123550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:59.124820 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:59.125200 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:59.125202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:59.125202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:59.125282 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:10:59.125494 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:10:59.125495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:10:59.125495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:10:59.125653 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:59.125766 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.125894 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.125973 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:10:59.127660 4641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:10:59.129379 4641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:10:59.136160 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.132066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:35809 (local address 127.0.0.1:58028)
0811 07:10:59.132467 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.132477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.132532 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.132911 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.132912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.132914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:59.132959 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.132961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.133922 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.133923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.135151 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.135153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.135847 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.135848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.135933 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.135948 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.136023 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.136080 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":4,"thread_start_us":70,"threads_started":1}
I20260811 07:10:59.136178 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.132412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.3.236.131:35809)
0811 07:10:59.132680 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.132685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.132695 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:10:59.132735 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.132736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.132750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.132828 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:59.133032 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.133782 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.135251 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.135752 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.135847 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.135910 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.136050 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.136091 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":141,"threads_started":1}
I20260811 07:10:59.136514 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.131961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.3.236.130:33687)
0811 07:10:59.132194 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.132200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.132212 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:10:59.132277 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.132279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.132297 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.132611 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:59.132696 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.133528 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.135270 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.135991 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.136065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.136118 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.136383 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.136436 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260811 07:10:59.136475 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.131788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:33687 (local address 127.0.0.1:53140)
0811 07:10:59.131972 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.132004 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.132098 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.132427 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.132430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.132433 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:59.132510 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.132513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.133635 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.133636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.135147 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.135151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.136153 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.136154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.136226 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.136230 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.136339 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.136396 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":79,"thread_start_us":116,"threads_started":1}
I20260811 07:10:59.137359 5267 tablet_service.cc:1467] Processing CreateTablet for tablet 194736043ff643af85a6c15fc7728efe (DEFAULT_TABLE table=test-table [id=04fa50e9a975422cbb4be2d088ab6b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:59.137355 5047 tablet_service.cc:1467] Processing CreateTablet for tablet 194736043ff643af85a6c15fc7728efe (DEFAULT_TABLE table=test-table [id=04fa50e9a975422cbb4be2d088ab6b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:59.137678 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.132239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:40849 (local address 127.0.0.1:57782)
0811 07:10:59.133685 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.133696 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.133759 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.134041 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.134041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.134043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:59.134080 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.134081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.135211 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.135212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.136363 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.136365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.137425 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.137426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.137494 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.137499 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.137568 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.137610 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1421,"thread_start_us":95,"threads_started":1}
I20260811 07:10:59.137701 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.132441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.3.236.129:40849)
0811 07:10:59.132780 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.132785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.133758 (+ 973us) server_negotiation.cc:407] Connection header received
0811 07:10:59.133847 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.133850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.133885 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.133959 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:59.134167 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.135104 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.136449 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.137323 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.137414 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.137474 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.137580 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.137622 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":115,"threads_started":1}
I20260811 07:10:59.137786 5047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194736043ff643af85a6c15fc7728efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:59.137795 5267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194736043ff643af85a6c15fc7728efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:59.138375 4838 tablet_service.cc:1467] Processing CreateTablet for tablet 194736043ff643af85a6c15fc7728efe (DEFAULT_TABLE table=test-table [id=04fa50e9a975422cbb4be2d088ab6b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:10:59.138698 4838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 194736043ff643af85a6c15fc7728efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:59.140805 5421 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Bootstrap starting.
I20260811 07:10:59.141323 5422 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Bootstrap starting.
I20260811 07:10:59.141570 5421 tablet_bootstrap.cc:654] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:59.142088 5423 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Bootstrap starting.
I20260811 07:10:59.142238 5422 tablet_bootstrap.cc:654] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:59.142227 5421 log.cc:824] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:59.142743 5423 tablet_bootstrap.cc:654] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:59.142833 5422 log.cc:824] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:59.143121 5421 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: No bootstrap required, opened a new log
I20260811 07:10:59.143224 5421 ts_tablet_manager.cc:1397] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:59.143389 5423 log.cc:824] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:59.143726 5422 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: No bootstrap required, opened a new log
I20260811 07:10:59.143802 5422 ts_tablet_manager.cc:1397] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:10:59.144170 5423 tablet_bootstrap.cc:492] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: No bootstrap required, opened a new log
I20260811 07:10:59.144258 5423 ts_tablet_manager.cc:1397] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:10:59.145340 5421 raft_consensus.cc:348] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.145565 5421 raft_consensus.cc:374] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:59.145607 5421 raft_consensus.cc:729] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f39f3b396684136a1a24a9ebf3480b7, State: Initialized, Role: FOLLOWER
I20260811 07:10:59.145753 5421 consensus_queue.cc:260] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.145920 5423 raft_consensus.cc:348] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.146051 5421 ts_tablet_manager.cc:1428] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:59.145989 5422 raft_consensus.cc:348] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.146109 5423 raft_consensus.cc:374] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:59.146153 5423 raft_consensus.cc:729] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eea1dfd3bbd4c98a4bf8648dd10e0a0, State: Initialized, Role: FOLLOWER
I20260811 07:10:59.146185 5391 heartbeater.cc:503] Master 127.3.236.190:35583 was elected leader, sending a full tablet report...
I20260811 07:10:59.146203 5422 raft_consensus.cc:374] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:59.146246 5422 raft_consensus.cc:729] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481644ff2caa42c398d97e1c7e09154e, State: Initialized, Role: FOLLOWER
I20260811 07:10:59.146260 5423 consensus_queue.cc:260] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.146358 5422 consensus_queue.cc:260] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.146565 5423 ts_tablet_manager.cc:1428] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:10:59.146674 5422 ts_tablet_manager.cc:1428] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:10:59.146726 4937 heartbeater.cc:503] Master 127.3.236.190:35583 was elected leader, sending a full tablet report...
I20260811 07:10:59.146831 5152 heartbeater.cc:503] Master 127.3.236.190:35583 was elected leader, sending a full tablet report...
W20260811 07:10:59.166659 5155 tablet.cc:2369] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:10:59.249145 4940 tablet.cc:2369] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:10:59.357973 5393 tablet.cc:2369] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:10:59.556914 5429 raft_consensus.cc:482] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:10:59.557076 5429 raft_consensus.cc:504] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.557528 5429 leader_election.cc:288] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481644ff2caa42c398d97e1c7e09154e (127.3.236.130:33687), 3eea1dfd3bbd4c98a4bf8648dd10e0a0 (127.3.236.129:40849)
I20260811 07:10:59.561796 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.558075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:40879 (local address 127.3.236.129:40849)
0811 07:10:59.558123 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.558125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.558255 (+ 130us) server_negotiation.cc:407] Connection header received
0811 07:10:59.558292 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.558293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.558306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.558364 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:59.558519 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.559231 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.560809 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.561414 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.561477 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.561485 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.561685 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.561725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:10:59.561796 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.557840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:38501 (local address 127.3.236.130:33687)
0811 07:10:59.557912 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.557915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.557927 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:10:59.558013 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.558014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.558028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.558091 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:10:59.558284 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.559092 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.560809 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.561394 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.561476 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.561485 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.561684 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.561725 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:10:59.561817 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.557762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:33687 (local address 127.3.236.131:38501)
0811 07:10:59.557826 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.557837 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.557917 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.558107 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.558108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.558111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:59.558199 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.558201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.559204 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.559205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.560696 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.560699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.561495 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.561496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.561593 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.561596 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.561668 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.561716 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":5}
I20260811 07:10:59.561825 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.558015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:40849 (local address 127.3.236.131:40879)
0811 07:10:59.558156 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.558169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.558238 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.558381 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.558382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.558384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:10:59.558443 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.558445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.559331 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.559332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.560696 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.560699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.561495 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.561496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.561590 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.561601 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.561668 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.561715 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":11,"thread_start_us":80,"threads_started":1}
I20260811 07:10:59.562268 5081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194736043ff643af85a6c15fc7728efe" candidate_uuid: "5f39f3b396684136a1a24a9ebf3480b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481644ff2caa42c398d97e1c7e09154e" is_pre_election: true
I20260811 07:10:59.562319 4867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194736043ff643af85a6c15fc7728efe" candidate_uuid: "5f39f3b396684136a1a24a9ebf3480b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" is_pre_election: true
I20260811 07:10:59.562433 5081 raft_consensus.cc:2454] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f39f3b396684136a1a24a9ebf3480b7 in term 0.
I20260811 07:10:59.562476 4867 raft_consensus.cc:2454] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f39f3b396684136a1a24a9ebf3480b7 in term 0.
I20260811 07:10:59.562695 5187 leader_election.cc:302] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eea1dfd3bbd4c98a4bf8648dd10e0a0, 5f39f3b396684136a1a24a9ebf3480b7; no voters:
I20260811 07:10:59.562883 5429 raft_consensus.cc:2781] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:10:59.562975 5429 raft_consensus.cc:482] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:10:59.563021 5429 raft_consensus.cc:3037] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:59.563807 5429 raft_consensus.cc:504] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.563953 5429 leader_election.cc:288] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [CANDIDATE]: Term 1 election: Requested vote from peers 481644ff2caa42c398d97e1c7e09154e (127.3.236.130:33687), 3eea1dfd3bbd4c98a4bf8648dd10e0a0 (127.3.236.129:40849)
I20260811 07:10:59.564170 5081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194736043ff643af85a6c15fc7728efe" candidate_uuid: "5f39f3b396684136a1a24a9ebf3480b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481644ff2caa42c398d97e1c7e09154e"
I20260811 07:10:59.564204 4867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194736043ff643af85a6c15fc7728efe" candidate_uuid: "5f39f3b396684136a1a24a9ebf3480b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0"
I20260811 07:10:59.564306 5081 raft_consensus.cc:3037] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:59.564316 4867 raft_consensus.cc:3037] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:59.565044 4867 raft_consensus.cc:2454] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f39f3b396684136a1a24a9ebf3480b7 in term 1.
I20260811 07:10:59.565160 5081 raft_consensus.cc:2454] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f39f3b396684136a1a24a9ebf3480b7 in term 1.
I20260811 07:10:59.565239 5187 leader_election.cc:302] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eea1dfd3bbd4c98a4bf8648dd10e0a0, 5f39f3b396684136a1a24a9ebf3480b7; no voters:
I20260811 07:10:59.565362 5429 raft_consensus.cc:2781] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:59.565536 5429 raft_consensus.cc:686] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f39f3b396684136a1a24a9ebf3480b7, State: Running, Role: LEADER
I20260811 07:10:59.565686 5429 consensus_queue.cc:237] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } }
I20260811 07:10:59.566588 4641 catalog_manager.cc:5495] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f39f3b396684136a1a24a9ebf3480b7 (127.3.236.131). New cstate: current_term: 1 leader_uuid: "5f39f3b396684136a1a24a9ebf3480b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f39f3b396684136a1a24a9ebf3480b7" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 35809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 } health_report { overall_health: UNKNOWN } } }
I20260811 07:10:59.575446 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.573188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.3.236.131:35809)
0811 07:10:59.573253 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.573255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.573264 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:10:59.573294 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.573295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.573310 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.573365 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:10:59.573480 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.573933 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.574662 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.574838 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.574897 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.575117 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:10:59.575210 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:10:59.575272 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.575315 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.575333 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:10:59.577312 5081 raft_consensus.cc:1261] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 1 FOLLOWER]: Refusing update from remote peer 5f39f3b396684136a1a24a9ebf3480b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:59.577312 4867 raft_consensus.cc:1261] T 194736043ff643af85a6c15fc7728efe P 3eea1dfd3bbd4c98a4bf8648dd10e0a0 [term 1 FOLLOWER]: Refusing update from remote peer 5f39f3b396684136a1a24a9ebf3480b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:10:59.577653 5429 consensus_queue.cc:1035] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "481644ff2caa42c398d97e1c7e09154e" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:59.577816 5434 consensus_queue.cc:1035] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eea1dfd3bbd4c98a4bf8648dd10e0a0" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 40849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:10:59.580610 5439 mvcc.cc:205] Tried to move back new op lower bound from 7317226535225982976 to 7317226535181725696. Current Snapshot: MvccSnapshot[applied={T|T < 7317226535225982976}]
I20260811 07:10:59.583101 4639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:59.583214 4639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:10:59.583984 4639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 830343183d6a4b3cbf383d400e7f7775: Sending DeleteTablet for 3 replicas of tablet 194736043ff643af85a6c15fc7728efe
I20260811 07:10:59.584411 5047 tablet_service.cc:1514] Processing DeleteTablet for tablet 194736043ff643af85a6c15fc7728efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:53140
I20260811 07:10:59.584465 5267 tablet_service.cc:1514] Processing DeleteTablet for tablet 194736043ff643af85a6c15fc7728efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:58028
I20260811 07:10:59.584494 4838 tablet_service.cc:1514] Processing DeleteTablet for tablet 194736043ff643af85a6c15fc7728efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:10:59 UTC) from {username='kudu', principal='kudu/127.3.236.190@KRBTEST.COM'} at 127.0.0.1:57782
I20260811 07:10:59.584731 5454 tablet_replica.cc:330] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: stopping tablet replica
I20260811 07:10:59.584844 5454 raft_consensus.cc:2229] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:10:59.584968 5454 raft_consensus.cc:2258] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:59.584972 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4735
I20260811 07:10:59.585868 5454 ts_tablet_manager.cc:1905] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:59.587733 5454 ts_tablet_manager.cc:1918] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:59.587850 5454 log.cc:1196] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-1/wal/wals/194736043ff643af85a6c15fc7728efe
I20260811 07:10:59.588155 5454 ts_tablet_manager.cc:1939] T 194736043ff643af85a6c15fc7728efe P 481644ff2caa42c398d97e1c7e09154e: Deleting consensus metadata
I20260811 07:10:59.588655 4619 catalog_manager.cc:4841] TS 481644ff2caa42c398d97e1c7e09154e (127.3.236.130:33687): tablet 194736043ff643af85a6c15fc7728efe (table test-table [id=04fa50e9a975422cbb4be2d088ab6b76]) successfully deleted
I20260811 07:10:59.595819 5455 tablet_replica.cc:330] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: stopping tablet replica
I20260811 07:10:59.596036 5455 raft_consensus.cc:2229] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:10:59.596192 5455 raft_consensus.cc:2258] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:10:59.597376 5455 ts_tablet_manager.cc:1905] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:10:59.599056 5455 ts_tablet_manager.cc:1918] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:10:59.599179 5455 log.cc:1196] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/194736043ff643af85a6c15fc7728efe
I20260811 07:10:59.599467 5455 ts_tablet_manager.cc:1939] T 194736043ff643af85a6c15fc7728efe P 5f39f3b396684136a1a24a9ebf3480b7: Deleting consensus metadata
I20260811 07:10:59.599943 4619 catalog_manager.cc:4841] TS 5f39f3b396684136a1a24a9ebf3480b7 (127.3.236.131:35809): tablet 194736043ff643af85a6c15fc7728efe (table test-table [id=04fa50e9a975422cbb4be2d088ab6b76]) successfully deleted
W20260811 07:10:59.602260 4619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:40849 (error 108)
I20260811 07:10:59.602541 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4945
I20260811 07:10:59.612087 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.611908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:40849 (local address 127.0.0.1:57796)
0811 07:10:59.611991 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.612049 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:40849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 07:10:59.612257 4619 catalog_manager.cc:4584] TS 3eea1dfd3bbd4c98a4bf8648dd10e0a0 (127.3.236.129:40849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 194736043ff643af85a6c15fc7728efe: Network error: Client connection negotiation failed: client connection to 127.3.236.129:40849: connect: Connection refused (error 111)
I20260811 07:10:59.617673 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5161
I20260811 07:10:59.629818 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 4590
2026-08-11T07:10:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2632 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T07:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:10:59Z Disabled control of system clock
I20260811 07:10:59.667927 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42087
--webserver_interface=127.3.236.190
--webserver_port=0
--builtin_ntp_servers=127.3.236.148:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:59.750813 5465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:59.751065 5465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:59.751085 5465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:59.752208 5465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:10:59.752247 5465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:59.752321 5465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:59.752352 5465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:10:59.752367 5465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:10:59.753726 5465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:59.753926 5465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:59.754173 5465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:59.759327 5472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:59.759325 5470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:59.759346 5480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:59.759434 5465 server_base.cc:1034] running on GCE node
I20260811 07:10:59.760156 5465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:59.760484 5465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:59.761636 5465 hybrid_clock.cc:648] HybridClock initialized: now 1786432259761614 us; error 42 us; skew 500 ppm
I20260811 07:10:59.762893 5465 webserver.cc:460] Webserver started at http://127.3.236.190:39289/ using document root <none> and password file <none>
I20260811 07:10:59.763128 5465 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:59.763172 5465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:59.763262 5465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:59.764343 5465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "96b904c4dc664e8caaba5788ac59c5a5"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8cdeb741310faf45a7c82a270a55f99d"
server_key_iv: "83ee04364edc54e60170bf7615d9b8e6"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.764669 5465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "96b904c4dc664e8caaba5788ac59c5a5"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8cdeb741310faf45a7c82a270a55f99d"
server_key_iv: "83ee04364edc54e60170bf7615d9b8e6"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.766031 5465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:59.766764 5488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.766986 5465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:10:59.767098 5465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "96b904c4dc664e8caaba5788ac59c5a5"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8cdeb741310faf45a7c82a270a55f99d"
server_key_iv: "83ee04364edc54e60170bf7615d9b8e6"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.767180 5465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:59.782770 5465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:59.783105 5465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:59.783257 5465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:59.787578 5465 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:42087
I20260811 07:10:59.787617 5587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:42087 every 8 connection(s)
I20260811 07:10:59.787966 5465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:10:59.789036 5590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:10:59.791244 5590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: Bootstrap starting.
I20260811 07:10:59.791787 5590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: Neither blocks nor log segments found. Creating new log.
I20260811 07:10:59.792049 5590 log.cc:824] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:10:59.792536 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5465
I20260811 07:10:59.792647 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:10:59.792836 5590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: No bootstrap required, opened a new log
I20260811 07:10:59.792860 4018 external_mini_cluster.cc:1428] Setting key a6f49d6b1b25856f8de2000d207fd3b7
I20260811 07:10:59.794721 5590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } }
I20260811 07:10:59.794900 5590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:10:59.794943 5590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b904c4dc664e8caaba5788ac59c5a5, State: Initialized, Role: FOLLOWER
I20260811 07:10:59.795080 5590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } }
I20260811 07:10:59.795172 5590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:10:59.795238 5590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:10:59.795286 5590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:10:59.795819 5590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } }
I20260811 07:10:59.795907 5590 leader_election.cc:302] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b904c4dc664e8caaba5788ac59c5a5; no voters:
I20260811 07:10:59.796090 5590 leader_election.cc:288] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:10:59.796123 5597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:10:59.796231 5597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [term 1 LEADER]: Becoming Leader. State: Replica: 96b904c4dc664e8caaba5788ac59c5a5, State: Running, Role: LEADER
I20260811 07:10:59.796380 5590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:10:59.796397 5597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } }
I20260811 07:10:59.796855 5597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96b904c4dc664e8caaba5788ac59c5a5. Latest consensus state: current_term: 1 leader_uuid: "96b904c4dc664e8caaba5788ac59c5a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } } }
I20260811 07:10:59.796864 5599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b904c4dc664e8caaba5788ac59c5a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b904c4dc664e8caaba5788ac59c5a5" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42087 } } }
I20260811 07:10:59.797150 5599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:59.797060 5597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:10:59.797428 5612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:10:59.798030 5612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:10:59.799961 5612 catalog_manager.cc:1347] Generated new cluster ID: 462469be19ca47ad92e011e215457dc5
I20260811 07:10:59.800062 5612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:10:59.800427 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.793563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.3.236.190:42087)
0811 07:10:59.793784 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.793791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.793815 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:10:59.794690 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.794697 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.794758 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.794873 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:59.795150 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.797018 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.798211 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.798707 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.798793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.799586 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:59.799589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:59.799589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:59.799797 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:59.799843 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:59.799857 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:59.799937 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.800082 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.800189 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260811 07:10:59.813715 5612 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:10:59.814239 5612 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:10:59.822086 5612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: Generated new TSK 0
I20260811 07:10:59.822302 5612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:10:59.830025 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.129:0
--local_ip_for_outbound_sockets=127.3.236.129
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--builtin_ntp_servers=127.3.236.148:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:10:59.924927 5629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:10:59.925164 5629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:10:59.925199 5629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:10:59.926384 5629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:10:59.926453 5629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:10:59.926496 5629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.129
I20260811 07:10:59.927830 5629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.236.129
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.129
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:10:59.928104 5629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:10:59.928426 5629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:10:59.934733 5635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:10:59.934787 5638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:59.935041 5629 server_base.cc:1034] running on GCE node
W20260811 07:10:59.934756 5640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:10:59.935336 5629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:10:59.935604 5629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:10:59.936767 5629 hybrid_clock.cc:648] HybridClock initialized: now 1786432259936751 us; error 31 us; skew 500 ppm
I20260811 07:10:59.938182 5629 webserver.cc:460] Webserver started at http://127.3.236.129:34123/ using document root <none> and password file <none>
I20260811 07:10:59.938445 5629 fs_manager.cc:359] Metadata directory not provided
I20260811 07:10:59.938501 5629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:10:59.938624 5629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:10:59.940028 5629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data/instance:
uuid: "7cee82a72b2442e4bd49df7be2657ea4"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8c2f3c54b15e0c5eb7b853dd15441845"
server_key_iv: "1e280fe9544c89a66f4698d379d792e7"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.940518 5629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance:
uuid: "7cee82a72b2442e4bd49df7be2657ea4"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8c2f3c54b15e0c5eb7b853dd15441845"
server_key_iv: "1e280fe9544c89a66f4698d379d792e7"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.942118 5629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:10:59.942960 5647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.943179 5629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:10:59.943274 5629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal
uuid: "7cee82a72b2442e4bd49df7be2657ea4"
format_stamp: "Formatted at 2026-08-11 07:10:59 on dist-test-slave-2d19"
server_key: "8c2f3c54b15e0c5eb7b853dd15441845"
server_key_iv: "1e280fe9544c89a66f4698d379d792e7"
server_key_version: "encryptionkey@0"
I20260811 07:10:59.943343 5629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:10:59.952562 5629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:10:59.952809 5629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:10:59.952914 5629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:10:59.953092 5629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:10:59.953361 5629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:10:59.953392 5629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.953437 5629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:10:59.953459 5629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:10:59.959542 5629 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.129:38135
I20260811 07:10:59.959978 5629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/data/info.pb
I20260811 07:10:59.960054 5847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.129:38135 every 8 connection(s)
I20260811 07:10:59.965272 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.960797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.129:58961 (local address 127.3.236.190:42087)
0811 07:10:59.960848 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:10:59.960850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:10:59.961079 (+ 229us) server_negotiation.cc:407] Connection header received
0811 07:10:59.961560 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:10:59.961561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:10:59.961575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:10:59.961629 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:10:59.962318 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.962884 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.963881 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.964112 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.964186 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.965025 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:10:59.965026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:10:59.965027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:10:59.965031 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:10:59.965046 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:10:59.965050 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:10:59.965096 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 07:10:59.965185 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 07:10:59.965211 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:10:59.965516 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:10:59.960713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42087 (local address 127.3.236.129:58961)
0811 07:10:59.960938 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 07:10:59.960972 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 07:10:59.961463 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:10:59.961685 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:10:59.961688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:10:59.961696 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:10:59.962198 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.962202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.962980 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.962980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.963793 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:10:59.963795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:10:59.964198 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:10:59.964199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:10:59.964374 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:10:59.964881 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:10:59.964881 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:10:59.964910 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:10:59.964933 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:10:59.965112 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:10:59.965117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:10:59.965120 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:10:59.965174 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 07:10:59.965254 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260811 07:10:59.966065 5849 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42087
I20260811 07:10:59.966146 5849 heartbeater.cc:464] Registering TS with master...
I20260811 07:10:59.966291 5849 heartbeater.cc:511] Master 127.3.236.190:42087 requested a full tablet report, sending...
I20260811 07:10:59.966758 5523 ts_manager.cc:194] Registered new tserver with Master: 7cee82a72b2442e4bd49df7be2657ea4 (127.3.236.129:38135)
I20260811 07:10:59.967350 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5629
I20260811 07:10:59.967351 5523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.129:58961
I20260811 07:10:59.967458 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-0/wal/instance
I20260811 07:10:59.967694 4018 external_mini_cluster.cc:1428] Setting key a605167e9b7426749d9279f73f6e326f
I20260811 07:10:59.968694 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.130:0
--local_ip_for_outbound_sockets=127.3.236.130
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--builtin_ntp_servers=127.3.236.148:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:11:00.053082 5854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:00.053293 5854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:00.053316 5854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:00.054479 5854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:00.054526 5854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:00.054553 5854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.130
I20260811 07:11:00.055900 5854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.236.130
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.130
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:00.056159 5854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:00.056506 5854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:00.061645 5864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:00.061648 5863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:00.061669 5868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:00.061985 5854 server_base.cc:1034] running on GCE node
I20260811 07:11:00.062237 5854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:00.062485 5854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:00.063653 5854 hybrid_clock.cc:648] HybridClock initialized: now 1786432260063633 us; error 33 us; skew 500 ppm
I20260811 07:11:00.065016 5854 webserver.cc:460] Webserver started at http://127.3.236.130:45317/ using document root <none> and password file <none>
I20260811 07:11:00.065303 5854 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:00.065379 5854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:00.065542 5854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:00.066653 5854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data/instance:
uuid: "afcb60dc350f45e5afc61e3f516a5e78"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "0f2352562ae81173af34650c06b85bc7"
server_key_iv: "898639c9d381cc1a91b1c773b8974272"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.067029 5854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance:
uuid: "afcb60dc350f45e5afc61e3f516a5e78"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "0f2352562ae81173af34650c06b85bc7"
server_key_iv: "898639c9d381cc1a91b1c773b8974272"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.068485 5854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:00.069303 5876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.069602 5854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:00.069701 5854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal
uuid: "afcb60dc350f45e5afc61e3f516a5e78"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "0f2352562ae81173af34650c06b85bc7"
server_key_iv: "898639c9d381cc1a91b1c773b8974272"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.069792 5854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:00.082448 5854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:00.082756 5854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:00.082911 5854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:00.083163 5854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:11:00.083503 5854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:11:00.083559 5854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.083602 5854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:11:00.083626 5854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.091624 5854 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.130:36653
I20260811 07:11:00.091830 6085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.130:36653 every 8 connection(s)
I20260811 07:11:00.092150 5854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/data/info.pb
I20260811 07:11:00.095503 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 5854
I20260811 07:11:00.095618 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-1/wal/instance
I20260811 07:11:00.095875 4018 external_mini_cluster.cc:1428] Setting key 2509787c00c23b59851e4f262c9271ed
I20260811 07:11:00.098348 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.236.131:0
--local_ip_for_outbound_sockets=127.3.236.131
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--builtin_ntp_servers=127.3.236.148:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:11:00.102052 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.095556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:54937 (local address 127.3.236.190:42087)
0811 07:11:00.095614 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.095617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.095811 (+ 194us) server_negotiation.cc:407] Connection header received
0811 07:11:00.096854 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.096856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.096872 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.096934 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:00.098174 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.098962 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.100503 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.100701 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.100782 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.101719 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:00.101721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:00.101721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:00.101726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:00.101743 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:00.101748 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:00.101822 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.101934 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.101969 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 07:11:00.102295 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.093082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42087 (local address 127.3.236.130:54937)
0811 07:11:00.095452 (+ 2370us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.095671 (+ 219us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.096707 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.097197 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.097202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.097216 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:00.098063 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.098071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.099105 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.099105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.100414 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.100419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.100797 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.100799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.100968 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.101545 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:11:00.101545 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:00.101585 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:11:00.101617 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:00.101839 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:00.101847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:00.101850 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.101937 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.102070 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2268,"thread_start_us":2205,"threads_started":1}
I20260811 07:11:00.102787 6087 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42087
I20260811 07:11:00.102897 6087 heartbeater.cc:464] Registering TS with master...
I20260811 07:11:00.103093 6087 heartbeater.cc:511] Master 127.3.236.190:42087 requested a full tablet report, sending...
I20260811 07:11:00.103542 5523 ts_manager.cc:194] Registered new tserver with Master: afcb60dc350f45e5afc61e3f516a5e78 (127.3.236.130:36653)
I20260811 07:11:00.103919 5523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.130:54937
W20260811 07:11:00.184000 6090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:00.184214 6090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:00.184237 6090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:00.185448 6090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:00.185491 6090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:00.185516 6090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.236.131
I20260811 07:11:00.186820 6090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.236.131
--webserver_port=0
--tserver_master_addrs=127.3.236.190:42087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.236.131
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:00.187012 6090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:00.187270 6090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:00.192410 6101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:00.192493 6102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:00.192410 6106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:00.192687 6090 server_base.cc:1034] running on GCE node
I20260811 07:11:00.192919 6090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:00.193145 6090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:00.194294 6090 hybrid_clock.cc:648] HybridClock initialized: now 1786432260194269 us; error 51 us; skew 500 ppm
I20260811 07:11:00.195659 6090 webserver.cc:460] Webserver started at http://127.3.236.131:45319/ using document root <none> and password file <none>
I20260811 07:11:00.195889 6090 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:00.195932 6090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:00.196077 6090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:00.197417 6090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data/instance:
uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "f568e84c65bda56b5a7efb8a859b2467"
server_key_iv: "e01b1cb664011d14970eddb37055fd12"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.197803 6090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance:
uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "f568e84c65bda56b5a7efb8a859b2467"
server_key_iv: "e01b1cb664011d14970eddb37055fd12"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.199285 6090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:00.200048 6115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.200300 6090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:00.200417 6090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal
uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84"
format_stamp: "Formatted at 2026-08-11 07:11:00 on dist-test-slave-2d19"
server_key: "f568e84c65bda56b5a7efb8a859b2467"
server_key_iv: "e01b1cb664011d14970eddb37055fd12"
server_key_version: "encryptionkey@0"
I20260811 07:11:00.200511 6090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:00.222458 6090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:00.222826 6090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:00.223013 6090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:00.223289 6090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:11:00.223667 6090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:11:00.223722 6090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.223755 6090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:11:00.223784 6090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:00.230132 6090 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.131:33181
I20260811 07:11:00.230172 6307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.131:33181 every 8 connection(s)
I20260811 07:11:00.230556 6090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/data/info.pb
I20260811 07:11:00.235420 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6090
I20260811 07:11:00.235585 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal/instance
I20260811 07:11:00.235833 4018 external_mini_cluster.cc:1428] Setting key df42c2664f978f417054d1a0afb10e4d
I20260811 07:11:00.236502 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.231382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.131:43267 (local address 127.3.236.190:42087)
0811 07:11:00.231430 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.231432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.231672 (+ 240us) server_negotiation.cc:407] Connection header received
0811 07:11:00.232364 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.232366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.232392 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.232446 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:00.233203 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.233723 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.234754 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.234955 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.235045 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.236061 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:00.236062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:00.236063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:00.236069 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:00.236087 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:00.236091 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:00.236174 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.236371 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.236409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:00.236775 6310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.231339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:42087 (local address 127.3.236.131:43267)
0811 07:11:00.231538 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.231576 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.232186 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.232546 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.232550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.232558 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:00.233096 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.233101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.233831 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.233832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.234655 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.234657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.235060 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.235062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.235202 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.235803 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:11:00.235804 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:00.235845 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:11:00.235879 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:00.236218 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:00.236227 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:00.236231 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.236372 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.236500 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260811 07:11:00.237218 6308 heartbeater.cc:347] Connected to a master server at 127.3.236.190:42087
I20260811 07:11:00.237327 6308 heartbeater.cc:464] Registering TS with master...
I20260811 07:11:00.237533 6308 heartbeater.cc:511] Master 127.3.236.190:42087 requested a full tablet report, sending...
I20260811 07:11:00.237896 5523 ts_manager.cc:194] Registered new tserver with Master: a8ffbf8c1e084abdbd81bd85f8a57f84 (127.3.236.131:33181)
I20260811 07:11:00.238276 5523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.236.131:43267
I20260811 07:11:00.247898 4018 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:11:00.252132 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.249422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.3.236.190:42087)
0811 07:11:00.249510 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.249512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.249572 (+ 60us) server_negotiation.cc:407] Connection header received
0811 07:11:00.249621 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.249622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.249631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.249679 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:00.249831 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.250309 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.251037 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.251209 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.251255 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.251856 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:00.251858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:00.251858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:00.251862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:00.251897 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:00.251901 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:00.251945 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.252025 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.252045 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:11:00.253630 5523 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:11:00.255157 5523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:11:00.262051 6328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:36653 (local address 127.0.0.1:50658)
0811 07:11:00.257818 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.257828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.257870 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.258315 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.258316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.258318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:11:00.258351 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.258352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259672 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.259674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.260801 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.260804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.261734 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.261796 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.261808 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.261902 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.261958 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":71,"thread_start_us":72,"threads_started":1}
I20260811 07:11:00.262180 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:33181 (local address 127.0.0.1:45504)
0811 07:11:00.257574 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.257607 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.257684 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.258186 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.258189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.258191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:11:00.258266 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.258268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259675 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.259676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.261020 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.261023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.261909 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.261962 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.261966 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.262064 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.262110 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":113,"thread_start_us":78,"threads_started":1}
I20260811 07:11:00.262475 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.3.236.130:36653)
0811 07:11:00.258017 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.258023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.258034 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:11:00.258077 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.258079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.258094 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.258215 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:11:00.258436 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259557 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.260905 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.261636 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261715 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.261813 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.262322 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.262389 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
I20260811 07:11:00.262751 6326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.3.236.131:33181)
0811 07:11:00.257993 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.257998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.258009 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:11:00.258051 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.258053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.258071 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.258349 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:11:00.258412 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259578 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261113 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.261815 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.261975 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.262078 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.262413 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":121,"threads_started":1}
I20260811 07:11:00.263324 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38135 (local address 127.0.0.1:49932)
0811 07:11:00.258258 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.258269 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.258314 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.258634 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.258635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.258637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:11:00.258682 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.258683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259598 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.259599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.261021 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.261023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.263042 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.263043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.263110 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.263114 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.263207 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.263252 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":67,"thread_start_us":61,"threads_started":1}
I20260811 07:11:00.263345 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.257972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.3.236.129:38135)
0811 07:11:00.258475 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.258478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.258487 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:11:00.258526 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.258527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.258538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.258624 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:11:00.258737 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.259509 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.261872 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.262826 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.262901 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.263003 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.263229 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.263266 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":138,"threads_started":1}
I20260811 07:11:00.263239 5963 tablet_service.cc:1467] Processing CreateTablet for tablet 31c69615d6a44750a48648f66854f3ff (DEFAULT_TABLE table=test-table [id=742581babe5445d69b0d7edd5628b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:11:00.263409 6193 tablet_service.cc:1467] Processing CreateTablet for tablet 31c69615d6a44750a48648f66854f3ff (DEFAULT_TABLE table=test-table [id=742581babe5445d69b0d7edd5628b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:11:00.263669 5963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c69615d6a44750a48648f66854f3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:00.263701 6193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c69615d6a44750a48648f66854f3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:00.264056 5736 tablet_service.cc:1467] Processing CreateTablet for tablet 31c69615d6a44750a48648f66854f3ff (DEFAULT_TABLE table=test-table [id=742581babe5445d69b0d7edd5628b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:11:00.264425 5736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c69615d6a44750a48648f66854f3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:00.267243 6338 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: Bootstrap starting.
I20260811 07:11:00.267488 6341 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Bootstrap starting.
I20260811 07:11:00.268110 6341 tablet_bootstrap.cc:654] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:00.268415 6339 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Bootstrap starting.
I20260811 07:11:00.268545 6341 log.cc:824] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:00.268939 6338 tablet_bootstrap.cc:654] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:00.269138 6339 tablet_bootstrap.cc:654] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:00.269309 6338 log.cc:824] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:00.269469 6341 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: No bootstrap required, opened a new log
I20260811 07:11:00.269491 6339 log.cc:824] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:00.269580 6341 ts_tablet_manager.cc:1397] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:11:00.270237 6339 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: No bootstrap required, opened a new log
I20260811 07:11:00.270247 6338 tablet_bootstrap.cc:492] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: No bootstrap required, opened a new log
I20260811 07:11:00.270310 6339 ts_tablet_manager.cc:1397] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:11:00.270310 6338 ts_tablet_manager.cc:1397] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 07:11:00.271787 6341 raft_consensus.cc:348] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.271983 6341 raft_consensus.cc:374] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:00.272037 6341 raft_consensus.cc:729] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcb60dc350f45e5afc61e3f516a5e78, State: Initialized, Role: FOLLOWER
I20260811 07:11:00.272157 6341 consensus_queue.cc:260] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.272456 6341 ts_tablet_manager.cc:1428] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 07:11:00.273049 6338 raft_consensus.cc:348] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.273077 6339 raft_consensus.cc:348] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.273224 6339 raft_consensus.cc:374] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:00.273224 6338 raft_consensus.cc:374] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:00.273262 6338 raft_consensus.cc:729] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cee82a72b2442e4bd49df7be2657ea4, State: Initialized, Role: FOLLOWER
I20260811 07:11:00.273262 6339 raft_consensus.cc:729] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ffbf8c1e084abdbd81bd85f8a57f84, State: Initialized, Role: FOLLOWER
I20260811 07:11:00.273379 6338 consensus_queue.cc:260] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.273379 6339 consensus_queue.cc:260] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.273653 6339 ts_tablet_manager.cc:1428] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 07:11:00.273653 6338 ts_tablet_manager.cc:1428] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 07:11:00.273727 5849 heartbeater.cc:503] Master 127.3.236.190:42087 was elected leader, sending a full tablet report...
I20260811 07:11:00.273773 6308 heartbeater.cc:503] Master 127.3.236.190:42087 was elected leader, sending a full tablet report...
I20260811 07:11:00.273780 6087 heartbeater.cc:503] Master 127.3.236.190:42087 was elected leader, sending a full tablet report...
I20260811 07:11:00.286610 6348 raft_consensus.cc:482] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:11:00.286715 6348 raft_consensus.cc:504] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.287111 6348 leader_election.cc:288] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ffbf8c1e084abdbd81bd85f8a57f84 (127.3.236.131:33181), 7cee82a72b2442e4bd49df7be2657ea4 (127.3.236.129:38135)
I20260811 07:11:00.290208 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.287531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:42457 (local address 127.3.236.129:38135)
0811 07:11:00.287579 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.287582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.287684 (+ 102us) server_negotiation.cc:407] Connection header received
0811 07:11:00.287727 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.287729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.287741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.287806 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:11:00.287946 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.288507 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289483 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.289831 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289932 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.289939 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.290092 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.290135 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:11:00.290221 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.287290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.131:33181 (local address 127.3.236.130:58201)
0811 07:11:00.287346 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.287359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.287404 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.287607 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.287608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.287610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:11:00.287684 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.287687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.288497 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.288498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.289362 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.289365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.289929 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.290023 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.290027 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.290104 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.290140 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":45}
I20260811 07:11:00.290230 6326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.287392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.236.130:58201 (local address 127.3.236.131:33181)
0811 07:11:00.287459 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.287461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.287470 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:11:00.287499 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.287499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.287516 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.287601 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:11:00.287766 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.288425 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289455 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.289833 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289917 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.289926 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.290123 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.290166 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:11:00.290233 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.287415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38135 (local address 127.3.236.130:42457)
0811 07:11:00.287606 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.287618 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:00.287681 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:00.287812 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:00.287814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:00.287815 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:11:00.287886 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.287888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.288588 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.288589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.289373 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:00.289375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.289937 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.289938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:00.290015 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.290019 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:11:00.290067 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 07:11:00.290128 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260811 07:11:00.290597 5775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c69615d6a44750a48648f66854f3ff" candidate_uuid: "afcb60dc350f45e5afc61e3f516a5e78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cee82a72b2442e4bd49df7be2657ea4" is_pre_election: true
I20260811 07:11:00.290652 6227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c69615d6a44750a48648f66854f3ff" candidate_uuid: "afcb60dc350f45e5afc61e3f516a5e78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" is_pre_election: true
I20260811 07:11:00.290763 5775 raft_consensus.cc:2454] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afcb60dc350f45e5afc61e3f516a5e78 in term 0.
I20260811 07:11:00.290828 6227 raft_consensus.cc:2454] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afcb60dc350f45e5afc61e3f516a5e78 in term 0.
I20260811 07:11:00.291031 5886 leader_election.cc:302] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cee82a72b2442e4bd49df7be2657ea4, afcb60dc350f45e5afc61e3f516a5e78; no voters:
I20260811 07:11:00.291198 6348 raft_consensus.cc:2781] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:11:00.291280 6348 raft_consensus.cc:482] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:11:00.291298 6348 raft_consensus.cc:3037] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:00.292075 6348 raft_consensus.cc:504] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.292218 6348 leader_election.cc:288] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [CANDIDATE]: Term 1 election: Requested vote from peers a8ffbf8c1e084abdbd81bd85f8a57f84 (127.3.236.131:33181), 7cee82a72b2442e4bd49df7be2657ea4 (127.3.236.129:38135)
I20260811 07:11:00.292445 6227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c69615d6a44750a48648f66854f3ff" candidate_uuid: "afcb60dc350f45e5afc61e3f516a5e78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84"
I20260811 07:11:00.292444 5775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c69615d6a44750a48648f66854f3ff" candidate_uuid: "afcb60dc350f45e5afc61e3f516a5e78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cee82a72b2442e4bd49df7be2657ea4"
I20260811 07:11:00.292527 6227 raft_consensus.cc:3037] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:00.292546 5775 raft_consensus.cc:3037] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:00.293458 6227 raft_consensus.cc:2454] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcb60dc350f45e5afc61e3f516a5e78 in term 1.
I20260811 07:11:00.293458 5775 raft_consensus.cc:2454] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afcb60dc350f45e5afc61e3f516a5e78 in term 1.
I20260811 07:11:00.293632 5886 leader_election.cc:302] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ffbf8c1e084abdbd81bd85f8a57f84, afcb60dc350f45e5afc61e3f516a5e78; no voters:
I20260811 07:11:00.293743 6348 raft_consensus.cc:2781] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:00.293888 6348 raft_consensus.cc:686] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 1 LEADER]: Becoming Leader. State: Replica: afcb60dc350f45e5afc61e3f516a5e78, State: Running, Role: LEADER
I20260811 07:11:00.294023 6348 consensus_queue.cc:237] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } }
I20260811 07:11:00.294884 5523 catalog_manager.cc:5495] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 reported cstate change: term changed from 0 to 1, leader changed from <none> to afcb60dc350f45e5afc61e3f516a5e78 (127.3.236.130). New cstate: current_term: 1 leader_uuid: "afcb60dc350f45e5afc61e3f516a5e78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afcb60dc350f45e5afc61e3f516a5e78" member_type: VOTER last_known_addr { host: "127.3.236.130" port: 36653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 } health_report { overall_health: UNKNOWN } } }
I20260811 07:11:00.308368 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.305905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.3.236.130:36653)
0811 07:11:00.305969 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:00.305972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:00.305980 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:11:00.306036 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:00.306037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:00.306058 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:00.306133 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:00.306282 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.306751 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.307518 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:00.307733 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:00.307812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:00.307979 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:00.308092 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:00.308180 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:00.308223 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 07:11:00.308241 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 07:11:00.310259 6227 raft_consensus.cc:1261] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 1 FOLLOWER]: Refusing update from remote peer afcb60dc350f45e5afc61e3f516a5e78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:00.310259 5775 raft_consensus.cc:1261] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 1 FOLLOWER]: Refusing update from remote peer afcb60dc350f45e5afc61e3f516a5e78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:00.310657 6348 consensus_queue.cc:1035] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cee82a72b2442e4bd49df7be2657ea4" member_type: VOTER last_known_addr { host: "127.3.236.129" port: 38135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:00.310806 6354 consensus_queue.cc:1035] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ffbf8c1e084abdbd81bd85f8a57f84" member_type: VOTER last_known_addr { host: "127.3.236.131" port: 33181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:00.314265 6360 mvcc.cc:205] Tried to move back new op lower bound from 7317226538227863552 to 7317226538165018624. Current Snapshot: MvccSnapshot[applied={T|T < 7317226538227863552}]
I20260811 07:11:00.315794 5523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:11:00.315899 5523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:11:00.316727 5523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96b904c4dc664e8caaba5788ac59c5a5: Sending DeleteTablet for 3 replicas of tablet 31c69615d6a44750a48648f66854f3ff
I20260811 07:11:00.317077 6193 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c69615d6a44750a48648f66854f3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:11:00 UTC) from {username='slave'} at 127.0.0.1:45504
I20260811 07:11:00.317080 5963 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c69615d6a44750a48648f66854f3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:11:00 UTC) from {username='slave'} at 127.0.0.1:50658
I20260811 07:11:00.317152 5736 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c69615d6a44750a48648f66854f3ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:11:00 UTC) from {username='slave'} at 127.0.0.1:49932
I20260811 07:11:00.317399 6377 tablet_replica.cc:330] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4: stopping tablet replica
I20260811 07:11:00.317503 6377 raft_consensus.cc:2229] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:11:00.317610 6377 raft_consensus.cc:2258] T 31c69615d6a44750a48648f66854f3ff P 7cee82a72b2442e4bd49df7be2657ea4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:11:00.317694 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5629
I20260811 07:11:00.324409 6376 tablet_replica.cc:330] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: stopping tablet replica
I20260811 07:11:00.324558 6376 raft_consensus.cc:2229] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:11:00.324682 6376 raft_consensus.cc:2258] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:11:00.324857 6374 tablet_replica.cc:330] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: stopping tablet replica
I20260811 07:11:00.324954 6374 raft_consensus.cc:2229] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:11:00.325076 6374 raft_consensus.cc:2258] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:11:00.326098 6376 ts_tablet_manager.cc:1905] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:11:00.327708 6376 ts_tablet_manager.cc:1918] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:11:00.327804 6376 log.cc:1196] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Deleting WAL directory at /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786432006544353-4018-0/minicluster-data/ts-2/wal/wals/31c69615d6a44750a48648f66854f3ff
I20260811 07:11:00.328094 6376 ts_tablet_manager.cc:1939] T 31c69615d6a44750a48648f66854f3ff P a8ffbf8c1e084abdbd81bd85f8a57f84: Deleting consensus metadata
I20260811 07:11:00.328616 5493 catalog_manager.cc:4841] TS a8ffbf8c1e084abdbd81bd85f8a57f84 (127.3.236.131:33181): tablet 31c69615d6a44750a48648f66854f3ff (table test-table [id=742581babe5445d69b0d7edd5628b2d8]) successfully deleted
W20260811 07:11:00.331616 5493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.236.129:38135 (error 108)
I20260811 07:11:00.331799 6374 ts_tablet_manager.cc:1905] T 31c69615d6a44750a48648f66854f3ff P afcb60dc350f45e5afc61e3f516a5e78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:11:00.332224 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.332091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.129:38135 (local address 127.0.0.1:49942)
0811 07:11:00.332146 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.332188 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:11:00.332228 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5854
W20260811 07:11:00.332475 5493 catalog_manager.cc:4584] TS 7cee82a72b2442e4bd49df7be2657ea4 (127.3.236.129:38135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31c69615d6a44750a48648f66854f3ff: Network error: Client connection negotiation failed: client connection to 127.3.236.129:38135: connect: Connection refused (error 111)
I20260811 07:11:00.343230 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6090
I20260811 07:11:00.343355 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:00.343212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.130:36653 (local address 127.0.0.1:50678)
0811 07:11:00.343277 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:00.343308 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.130:36653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 07:11:00.364892 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 5465
2026-08-11T07:11:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (738 ms)
[----------] 6 tests from EncryptionPolicyTest (9936 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:00 dist-test-slave-2d19 krb5kdc[6383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:00 dist-test-slave-2d19 krb5kdc[6383](info): set up 2 sockets
Aug 11 07:11:00 dist-test-slave-2d19 krb5kdc[6383](info): commencing operation
krb5kdc: starting...
W20260811 07:11:01.683429 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:01 dist-test-slave-2d19 krb5kdc[6383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:01.811434 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:01.898906 6408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:01.899147 6408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:01.899171 6408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:01.900275 6408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:01.900312 6408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:01.900326 6408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:01.900339 6408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:01.900352 6408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:01.901921 6408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:01.902129 6408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:01.902315 6408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:01.907411 6416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:01.907441 6418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:01.907507 6408 server_base.cc:1034] running on GCE node
W20260811 07:11:01.907424 6420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:01.907987 6408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:01.908367 6408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:01.909547 6408 hybrid_clock.cc:648] HybridClock initialized: now 1786432261909527 us; error 37 us; skew 500 ppm
Aug 11 07:11:01 dist-test-slave-2d19 krb5kdc[6383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:01.917518 6408 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:01.918004 6408 webserver.cc:460] Webserver started at http://127.0.0.1:43767/ using document root <none> and password file <none>
I20260811 07:11:01.918267 6408 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:01.918336 6408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:01.918494 6408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:01.919577 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7"
format_stamp: "Formatted at 2026-08-11 07:11:01 on dist-test-slave-2d19"
server_key: "553369044ecaedab7bb952526888ac40"
server_key_iv: "775bd26e7119239e70fc7d1b8cd713ea"
server_key_version: "encryptionkey@0"
I20260811 07:11:01.919948 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7"
format_stamp: "Formatted at 2026-08-11 07:11:01 on dist-test-slave-2d19"
server_key: "553369044ecaedab7bb952526888ac40"
server_key_iv: "775bd26e7119239e70fc7d1b8cd713ea"
server_key_version: "encryptionkey@0"
I20260811 07:11:01.921433 6408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:01.922238 6433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:01.922466 6408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:01.922590 6408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7"
format_stamp: "Formatted at 2026-08-11 07:11:01 on dist-test-slave-2d19"
server_key: "553369044ecaedab7bb952526888ac40"
server_key_iv: "775bd26e7119239e70fc7d1b8cd713ea"
server_key_version: "encryptionkey@0"
I20260811 07:11:01.922667 6408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:01.930977 6408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:01.933434 6408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:01.933609 6408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:01.938215 6408 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34441
I20260811 07:11:01.938275 6539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34441 every 8 connection(s)
I20260811 07:11:01.938683 6408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:01.939725 6545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:01.942106 6545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7: Bootstrap starting.
I20260811 07:11:01.942654 6545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:01.942888 6545 log.cc:824] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:01.943593 6545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7: No bootstrap required, opened a new log
I20260811 07:11:01.945276 6545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } }
I20260811 07:11:01.945417 6545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:01.945441 6545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c4cddbfd0f452ea57b4bd328a8d6b7, State: Initialized, Role: FOLLOWER
I20260811 07:11:01.945605 6545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } }
I20260811 07:11:01.945705 6545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:01.945731 6545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:01.945761 6545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:01.946264 6545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } }
I20260811 07:11:01.946349 6545 leader_election.cc:302] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21c4cddbfd0f452ea57b4bd328a8d6b7; no voters:
I20260811 07:11:01.946468 6545 leader_election.cc:288] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:01.946547 6549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:01.946738 6549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [term 1 LEADER]: Becoming Leader. State: Replica: 21c4cddbfd0f452ea57b4bd328a8d6b7, State: Running, Role: LEADER
I20260811 07:11:01.946774 6545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:01.946859 6549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } }
I20260811 07:11:01.947180 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6408
I20260811 07:11:01.947325 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:01.947556 4018 external_mini_cluster.cc:1428] Setting key 7f19432e64e0c7815193787842a2866a
I20260811 07:11:01.948005 6552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21c4cddbfd0f452ea57b4bd328a8d6b7. Latest consensus state: current_term: 1 leader_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } } }
I20260811 07:11:01.948122 6552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:01.948369 6551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c4cddbfd0f452ea57b4bd328a8d6b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34441 } } }
I20260811 07:11:01.948606 6551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:01.948348 6577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:01.949213 6577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:01.950809 6577 catalog_manager.cc:1347] Generated new cluster ID: cbef61256a5f407699fe3886ae464971
I20260811 07:11:01.950863 6577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:11:01 dist-test-slave-2d19 krb5kdc[6383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:11:01.956689 6577 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:01.957381 6577 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:01.958469 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:01.948373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33353 (local address 127.0.0.1:34441)
0811 07:11:01.948691 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:01.948702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:01.948743 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:11:01.948862 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:01.948868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:01.948911 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:01.949088 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:01.949820 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:01.951086 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:01.951860 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:01.952370 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:01.952494 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:01.954905 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:01.954909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:01.954909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:01.954928 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:01.956823 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:01.957240 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:01.957244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:01.957244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:01.957336 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:01.957628 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:01.957629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:01.957629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:01.957855 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:01.958057 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:01.958150 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:11:01.958258 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
I20260811 07:11:01.959455 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:01.963089 6577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21c4cddbfd0f452ea57b4bd328a8d6b7: Generated new TSK 0
I20260811 07:11:01.963253 6577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:01.967550 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:01.960917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44989 (local address 127.0.0.1:34441)
0811 07:11:01.960952 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:01.960955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:01.961164 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:11:01.961224 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:01.961225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:01.961236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:01.961290 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:01.961816 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:01.962354 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:01.963240 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:01.963528 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:01.963604 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:01.964887 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:01.964888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:01.964888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:01.964895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:01.966306 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:01.966688 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:01.966690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:01.966690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:01.966750 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:01.966981 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:01.966982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:01.966982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:01.967094 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:01.967205 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:01.967327 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:11:01.967439 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 07:11:01.969872 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6408
W20260811 07:11:01.984944 6381 connection.cc:535] client connection to 127.0.0.1:34441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34441): Connection reset by peer (error 104)
2026-08-11T07:11:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1600 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:02 dist-test-slave-2d19 krb5kdc[6596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:02 dist-test-slave-2d19 krb5kdc[6596](info): set up 2 sockets
Aug 11 07:11:02 dist-test-slave-2d19 krb5kdc[6596](info): commencing operation
krb5kdc: starting...
W20260811 07:11:03.265089 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:03 dist-test-slave-2d19 krb5kdc[6596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:03.391537 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:03.477911 6618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:03.478262 6618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:03.478304 6618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:03.480311 6618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:03.480378 6618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:03.480403 6618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:03.480423 6618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:03.480443 6618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:03.483054 6618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:03.483349 6618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:03.483630 6618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:03.489749 6625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:03.489768 6627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:03.489811 6629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:03.490103 6618 server_base.cc:1034] running on GCE node
I20260811 07:11:03.490417 6618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:03.490744 6618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:03.491894 6618 hybrid_clock.cc:648] HybridClock initialized: now 1786432263491857 us; error 46 us; skew 500 ppm
Aug 11 07:11:03 dist-test-slave-2d19 krb5kdc[6596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:03.494673 6618 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:03.499109 6618 webserver.cc:460] Webserver started at http://127.0.0.1:38115/ using document root <none> and password file <none>
I20260811 07:11:03.499377 6618 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:03.499454 6618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:03.499617 6618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:03.501084 6618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "91c89f19526a40488c9f145e9c87aa9c"
format_stamp: "Formatted at 2026-08-11 07:11:03 on dist-test-slave-2d19"
server_key: "e9b625d1ce15fcf332d8e730d4d1196f"
server_key_iv: "3dcd8172215d3eae88d856e2c931a386"
server_key_version: "encryptionkey@0"
I20260811 07:11:03.501624 6618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "91c89f19526a40488c9f145e9c87aa9c"
format_stamp: "Formatted at 2026-08-11 07:11:03 on dist-test-slave-2d19"
server_key: "e9b625d1ce15fcf332d8e730d4d1196f"
server_key_iv: "3dcd8172215d3eae88d856e2c931a386"
server_key_version: "encryptionkey@0"
I20260811 07:11:03.503257 6618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:11:03.504102 6645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:03.504289 6618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:03.504380 6618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "91c89f19526a40488c9f145e9c87aa9c"
format_stamp: "Formatted at 2026-08-11 07:11:03 on dist-test-slave-2d19"
server_key: "e9b625d1ce15fcf332d8e730d4d1196f"
server_key_iv: "3dcd8172215d3eae88d856e2c931a386"
server_key_version: "encryptionkey@0"
I20260811 07:11:03.504455 6618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:03.520413 6618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:03.523198 6618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:03.523356 6618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:03.528784 6618 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43533
I20260811 07:11:03.528857 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43533 every 8 connection(s)
I20260811 07:11:03.529198 6618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:03.530552 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:03.534389 6738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c: Bootstrap starting.
I20260811 07:11:03.535013 6738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:03.535269 6738 log.cc:824] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:03.535944 6738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c: No bootstrap required, opened a new log
I20260811 07:11:03.537670 6738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } }
I20260811 07:11:03.537813 6738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:03.537837 6738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c89f19526a40488c9f145e9c87aa9c, State: Initialized, Role: FOLLOWER
I20260811 07:11:03.537977 6738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } }
I20260811 07:11:03.538069 6738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:03.538092 6738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:03.538122 6738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:03.538622 6738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } }
I20260811 07:11:03.538710 6738 leader_election.cc:302] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c89f19526a40488c9f145e9c87aa9c; no voters:
I20260811 07:11:03.538839 6738 leader_election.cc:288] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:03.538916 6747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:03.539103 6738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:03.539130 6747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [term 1 LEADER]: Becoming Leader. State: Replica: 91c89f19526a40488c9f145e9c87aa9c, State: Running, Role: LEADER
I20260811 07:11:03.539337 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6618
I20260811 07:11:03.539350 6747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } }
I20260811 07:11:03.539494 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:03.539746 4018 external_mini_cluster.cc:1428] Setting key c39c0ffbe43fd6d918f2cd1afefb3345
I20260811 07:11:03.539765 6748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c89f19526a40488c9f145e9c87aa9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } } }
I20260811 07:11:03.539847 6748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:03.539899 6749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c89f19526a40488c9f145e9c87aa9c. Latest consensus state: current_term: 1 leader_uuid: "91c89f19526a40488c9f145e9c87aa9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c89f19526a40488c9f145e9c87aa9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43533 } } }
I20260811 07:11:03.539966 6749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:03.540663 6761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:03.541267 6761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:03.545387 6761 catalog_manager.cc:1347] Generated new cluster ID: 7f308c2b9d1d4cb58b11ecaed716c473
I20260811 07:11:03.545413 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:03.540608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53845 (local address 127.0.0.1:43533)
0811 07:11:03.540822 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:03.540833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:03.540869 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:11:03.540992 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:03.541000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:03.541099 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:03.541243 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:03.541445 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:03.542365 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:03.543157 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:03.543598 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:03.543723 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:03.544619 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:03.544624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:03.544624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:03.544837 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:03.544879 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:03.544897 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:03.544996 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:03.545153 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 07:11:03.545189 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260811 07:11:03.545444 6761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:03.546427 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:03.551484 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:03.547842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39137 (local address 127.0.0.1:43533)
0811 07:11:03.547873 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:03.547875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:03.548045 (+ 170us) server_negotiation.cc:407] Connection header received
0811 07:11:03.548076 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:03.548078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:03.548106 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:03.548186 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:03.548357 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:03.549100 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:03.550083 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:03.550436 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:03.550516 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:03.551220 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:03.551221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:03.551221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:03.551226 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:03.551241 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:03.551245 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:03.551311 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:03.551380 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 07:11:03.551394 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260811 07:11:03.563102 6761 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:03.563635 6761 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:03.568225 6761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c89f19526a40488c9f145e9c87aa9c: Generated new TSK 0
I20260811 07:11:03.568480 6761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:03.576588 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6618
W20260811 07:11:03.592231 6592 connection.cc:535] client connection to 127.0.0.1:43533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43533): Connection reset by peer (error 104)
2026-08-11T07:11:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:03 dist-test-slave-2d19 krb5kdc[6801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:03 dist-test-slave-2d19 krb5kdc[6801](info): set up 2 sockets
Aug 11 07:11:03 dist-test-slave-2d19 krb5kdc[6801](info): commencing operation
krb5kdc: starting...
W20260811 07:11:04.883725 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:04 dist-test-slave-2d19 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:05.011399 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:05.095069 6835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:05.095326 6835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:05.095367 6835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:05.096598 6835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:05.096661 6835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:05.096684 6835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:05.096707 6835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:05.096735 6835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:05.098199 6835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:05.098482 6835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:05.098717 6835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:05.106848 6845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:05.106881 6850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:05.107132 6846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:05.112890 6835 server_base.cc:1034] running on GCE node
I20260811 07:11:05.113371 6835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:05.113734 6835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:05.114923 6835 hybrid_clock.cc:648] HybridClock initialized: now 1786432265114906 us; error 30 us; skew 500 ppm
Aug 11 07:11:05 dist-test-slave-2d19 krb5kdc[6801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:05.117762 6835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:05.118299 6835 webserver.cc:460] Webserver started at http://127.0.0.1:45229/ using document root <none> and password file <none>
I20260811 07:11:05.118573 6835 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:05.118651 6835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:05.118820 6835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:05.120054 6835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "90ef39fe484d4275bf9d08500faa0e73"
format_stamp: "Formatted at 2026-08-11 07:11:05 on dist-test-slave-2d19"
server_key: "82f095c206a8425a7478d145e5241b7e"
server_key_iv: "a2885e23877e771d75077ad260720361"
server_key_version: "encryptionkey@0"
I20260811 07:11:05.120530 6835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "90ef39fe484d4275bf9d08500faa0e73"
format_stamp: "Formatted at 2026-08-11 07:11:05 on dist-test-slave-2d19"
server_key: "82f095c206a8425a7478d145e5241b7e"
server_key_iv: "a2885e23877e771d75077ad260720361"
server_key_version: "encryptionkey@0"
I20260811 07:11:05.122103 6835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:05.122941 6865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:05.123148 6835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:05.123274 6835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "90ef39fe484d4275bf9d08500faa0e73"
format_stamp: "Formatted at 2026-08-11 07:11:05 on dist-test-slave-2d19"
server_key: "82f095c206a8425a7478d145e5241b7e"
server_key_iv: "a2885e23877e771d75077ad260720361"
server_key_version: "encryptionkey@0"
I20260811 07:11:05.123366 6835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:05.127260 6835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:05.129391 6835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:05.129606 6835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:05.134080 6835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39591
I20260811 07:11:05.134137 6955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39591 every 8 connection(s)
I20260811 07:11:05.134522 6835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:05.135594 6957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:05.136989 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 6835
I20260811 07:11:05.137113 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:05.137321 4018 external_mini_cluster.cc:1428] Setting key a8dabfe82c8268705e52fb6fcf0e3154
I20260811 07:11:05.138574 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73: Bootstrap starting.
I20260811 07:11:05.139350 6957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:05.139726 6957 log.cc:824] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:05.140220 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:05.137920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49195 (local address 127.0.0.1:39591)
0811 07:11:05.138126 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:05.138135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:05.138164 (+ 29us) server_negotiation.cc:407] Connection header received
0811 07:11:05.138633 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:05.138640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:05.138694 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:05.138843 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:05.139609 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:05.139613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:05.139613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:05.139617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:05.139659 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:05.139671 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:05.139752 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:05.140024 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 07:11:05.140066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260811 07:11:05.140549 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73: No bootstrap required, opened a new log
I20260811 07:11:05.141453 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:05.143254 6957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260811 07:11:05.143457 6957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:05.143501 6957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ef39fe484d4275bf9d08500faa0e73, State: Initialized, Role: FOLLOWER
I20260811 07:11:05.143677 6957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260811 07:11:05.143795 6957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:05.143833 6957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:05.143874 6957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:05.144488 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:05.143206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56405 (local address 127.0.0.1:39591)
0811 07:11:05.143266 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:05.143268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:05.143413 (+ 145us) server_negotiation.cc:407] Connection header received
0811 07:11:05.143490 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:05.143492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:05.143499 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:05.143566 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:05.144226 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:05.144227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:05.144228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:05.144232 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:05.144247 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:05.144269 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:05.144332 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:05.144395 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 07:11:05.144408 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:11:05.144709 6957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260811 07:11:05.144816 6957 leader_election.cc:302] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90ef39fe484d4275bf9d08500faa0e73; no voters:
I20260811 07:11:05.144982 6957 leader_election.cc:288] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:05.145044 6977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:05.145222 6977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [term 1 LEADER]: Becoming Leader. State: Replica: 90ef39fe484d4275bf9d08500faa0e73, State: Running, Role: LEADER
I20260811 07:11:05.145282 6957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:05.145381 6977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260811 07:11:05.145802 6980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90ef39fe484d4275bf9d08500faa0e73. Latest consensus state: current_term: 1 leader_uuid: "90ef39fe484d4275bf9d08500faa0e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } } }
I20260811 07:11:05.145911 6980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:05.145792 6978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90ef39fe484d4275bf9d08500faa0e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90ef39fe484d4275bf9d08500faa0e73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } } }
I20260811 07:11:05.146065 6978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:05.146211 6990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:05.146744 6990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:05.148468 6990 catalog_manager.cc:1347] Generated new cluster ID: 7db1bc6d84ac462e8190b96a8c7e6a14
I20260811 07:11:05.148520 6990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:05.168160 6990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:05.168395 6990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:05.176891 6990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90ef39fe484d4275bf9d08500faa0e73: Generated new TSK 0
I20260811 07:11:05.177116 6990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:05.181763 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 6835
2026-08-11T07:11:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1606 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:05 dist-test-slave-2d19 krb5kdc[7009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:05 dist-test-slave-2d19 krb5kdc[7009](info): set up 2 sockets
Aug 11 07:11:05 dist-test-slave-2d19 krb5kdc[7009](info): commencing operation
krb5kdc: starting...
W20260811 07:11:06.467235 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:06.593191 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:06.674547 7031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:06.674808 7031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:06.674846 7031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:06.676002 7031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:06.676071 7031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:06.676095 7031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:06.676115 7031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:06.676136 7031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:06.677649 7031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:06.677919 7031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:06.678150 7031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:06.683014 7046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:06.683099 7050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:06.683015 7044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:06.683344 7031 server_base.cc:1034] running on GCE node
I20260811 07:11:06.683635 7031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:06.683914 7031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:06.685073 7031 hybrid_clock.cc:648] HybridClock initialized: now 1786432266685065 us; error 32 us; skew 500 ppm
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:06.687490 7031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:06.687969 7031 webserver.cc:460] Webserver started at http://127.0.0.1:40643/ using document root <none> and password file <none>
I20260811 07:11:06.688203 7031 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:06.688287 7031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:06.688452 7031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:06.689559 7031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "63f77e3d2aba4a15a9b8913440b8abfe"
format_stamp: "Formatted at 2026-08-11 07:11:06 on dist-test-slave-2d19"
server_key: "f4e00d375b54d8c3619e37d8f500e309"
server_key_iv: "0a14df2a14d29b0e4515c15196f352d8"
server_key_version: "encryptionkey@0"
I20260811 07:11:06.689927 7031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "63f77e3d2aba4a15a9b8913440b8abfe"
format_stamp: "Formatted at 2026-08-11 07:11:06 on dist-test-slave-2d19"
server_key: "f4e00d375b54d8c3619e37d8f500e309"
server_key_iv: "0a14df2a14d29b0e4515c15196f352d8"
server_key_version: "encryptionkey@0"
I20260811 07:11:06.691259 7031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:06.691990 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:06.692170 7031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:11:06.692291 7031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "63f77e3d2aba4a15a9b8913440b8abfe"
format_stamp: "Formatted at 2026-08-11 07:11:06 on dist-test-slave-2d19"
server_key: "f4e00d375b54d8c3619e37d8f500e309"
server_key_iv: "0a14df2a14d29b0e4515c15196f352d8"
server_key_version: "encryptionkey@0"
I20260811 07:11:06.692377 7031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:06.714098 7031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:06.716392 7031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:06.716559 7031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:06.720903 7031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32921
I20260811 07:11:06.720957 7151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32921 every 8 connection(s)
I20260811 07:11:06.721334 7031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:06.722327 7152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:06.724766 7152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe: Bootstrap starting.
I20260811 07:11:06.725309 7152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:06.725570 7152 log.cc:824] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:06.726258 7152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe: No bootstrap required, opened a new log
I20260811 07:11:06.727921 7152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260811 07:11:06.728067 7152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:06.728091 7152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f77e3d2aba4a15a9b8913440b8abfe, State: Initialized, Role: FOLLOWER
I20260811 07:11:06.728353 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7031
I20260811 07:11:06.728309 7152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260811 07:11:06.728416 7152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:06.728457 7152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:06.728482 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:06.728515 7152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:06.728698 4018 external_mini_cluster.cc:1428] Setting key deca271d717ef2e94bb41df2df2ac923
I20260811 07:11:06.729260 7152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260811 07:11:06.729437 7152 leader_election.cc:302] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63f77e3d2aba4a15a9b8913440b8abfe; no voters:
I20260811 07:11:06.729633 7152 leader_election.cc:288] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:06.729676 7158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:06.729785 7158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [term 1 LEADER]: Becoming Leader. State: Replica: 63f77e3d2aba4a15a9b8913440b8abfe, State: Running, Role: LEADER
I20260811 07:11:06.729913 7152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:06.729941 7158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } }
I20260811 07:11:06.730369 7158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63f77e3d2aba4a15a9b8913440b8abfe. Latest consensus state: current_term: 1 leader_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } } }
I20260811 07:11:06.730531 7158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:06.730542 7161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f77e3d2aba4a15a9b8913440b8abfe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32921 } } }
I20260811 07:11:06.730736 7161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:06.730911 7177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:06.731423 7177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:06.733759 7177 catalog_manager.cc:1347] Generated new cluster ID: df82630ca56c4c13913e235a5fe56b6b
I20260811 07:11:06.733827 7177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:11:06.740600 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:06.729183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51903 (local address 127.0.0.1:32921)
0811 07:11:06.729380 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:06.729389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:06.729427 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:11:06.729534 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:06.729556 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:06.729617 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:06.730634 (+ 1017us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:06.730756 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:06.732179 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:06.733166 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:06.733596 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:06.733677 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:06.736605 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:06.736609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:06.736610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:06.736624 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:06.738685 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:06.739089 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:06.739094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:06.739094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:06.739258 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:06.739489 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:06.739490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:06.739490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:06.739832 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:06.740028 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:06.740104 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 07:11:06.740276 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260811 07:11:06.741868 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:06.749080 7177 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:06.750123 7177 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:06.751276 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:06.743287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34979 (local address 127.0.0.1:32921)
0811 07:11:06.743355 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:06.743357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:06.743546 (+ 189us) server_negotiation.cc:407] Connection header received
0811 07:11:06.743616 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:06.743617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:06.743639 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:06.743684 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:06.744469 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:06.745214 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:06.746384 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:06.746699 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:06.746759 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:06.748316 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:06.748317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:06.748317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:06.748324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:06.750061 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:06.750480 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:06.750481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:06.750482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:06.750541 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:06.750755 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:06.750755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:06.750756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:06.750870 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:06.750964 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:06.751100 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 07:11:06.751187 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:11:06.761003 7177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63f77e3d2aba4a15a9b8913440b8abfe: Generated new TSK 0
I20260811 07:11:06.761220 7177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:06.766000 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7031
2026-08-11T07:11:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7210](info): set up 2 sockets
Aug 11 07:11:06 dist-test-slave-2d19 krb5kdc[7210](info): commencing operation
krb5kdc: starting...
W20260811 07:11:08.057430 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:08 dist-test-slave-2d19 krb5kdc[7210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:08.186514 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:08.268731 7234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:08.268934 7234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:08.268957 7234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:08.270037 7234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:08.270071 7234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:08.270087 7234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:08.270099 7234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:08.270112 7234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:08.271507 7234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:08.271714 7234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:08.271904 7234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:08.276973 7241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:08.276964 7242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:08.276978 7252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:08.277036 7234 server_base.cc:1034] running on GCE node
I20260811 07:11:08.277474 7234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:08.277788 7234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:08.278955 7234 hybrid_clock.cc:648] HybridClock initialized: now 1786432268278941 us; error 44 us; skew 500 ppm
Aug 11 07:11:08 dist-test-slave-2d19 krb5kdc[7210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:08.287973 7234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:08.288507 7234 webserver.cc:460] Webserver started at http://127.0.0.1:38803/ using document root <none> and password file <none>
I20260811 07:11:08.288779 7234 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:08.288849 7234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:08.289000 7234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:08.290091 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc"
format_stamp: "Formatted at 2026-08-11 07:11:08 on dist-test-slave-2d19"
server_key: "6e0b87f21ec201c19d885986992a7d34"
server_key_iv: "9ad68f5db64a11cafa26f05c75f9e12b"
server_key_version: "encryptionkey@0"
I20260811 07:11:08.290463 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc"
format_stamp: "Formatted at 2026-08-11 07:11:08 on dist-test-slave-2d19"
server_key: "6e0b87f21ec201c19d885986992a7d34"
server_key_iv: "9ad68f5db64a11cafa26f05c75f9e12b"
server_key_version: "encryptionkey@0"
I20260811 07:11:08.291850 7234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:08.292784 7265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:08.293004 7234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:11:08.293115 7234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc"
format_stamp: "Formatted at 2026-08-11 07:11:08 on dist-test-slave-2d19"
server_key: "6e0b87f21ec201c19d885986992a7d34"
server_key_iv: "9ad68f5db64a11cafa26f05c75f9e12b"
server_key_version: "encryptionkey@0"
I20260811 07:11:08.293190 7234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:08.330274 7234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:08.332461 7234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:08.332633 7234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:08.337153 7234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36881
I20260811 07:11:08.337226 7368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36881 every 8 connection(s)
I20260811 07:11:08.337605 7234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:08.338670 7369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:08.341116 7369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc: Bootstrap starting.
I20260811 07:11:08.341725 7369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:08.341979 7369 log.cc:824] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:08.342254 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7234
I20260811 07:11:08.342386 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:08.342581 4018 external_mini_cluster.cc:1428] Setting key 4421add834e82bebb7a273acb300571e
I20260811 07:11:08.342898 7369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc: No bootstrap required, opened a new log
I20260811 07:11:08.345955 7369 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260811 07:11:08.346129 7369 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:08.346159 7369 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c5ffd4dad94f0b9d4f2d8be7f62adc, State: Initialized, Role: FOLLOWER
I20260811 07:11:08.346280 7369 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260811 07:11:08.346371 7369 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:08.346421 7369 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:08.346477 7369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:08.347216 7369 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260811 07:11:08.347347 7369 leader_election.cc:302] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2c5ffd4dad94f0b9d4f2d8be7f62adc; no voters:
I20260811 07:11:08.347525 7369 leader_election.cc:288] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:08.347609 7377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:08.347755 7377 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [term 1 LEADER]: Becoming Leader. State: Replica: f2c5ffd4dad94f0b9d4f2d8be7f62adc, State: Running, Role: LEADER
I20260811 07:11:08.347783 7369 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:08.347844 7377 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260811 07:11:08.348145 7377 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2c5ffd4dad94f0b9d4f2d8be7f62adc. Latest consensus state: current_term: 1 leader_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260811 07:11:08.348220 7377 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:08.348559 7379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2c5ffd4dad94f0b9d4f2d8be7f62adc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260811 07:11:08.348691 7379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:08.348853 7386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:08.349402 7386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:08.349776 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:08.343241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38729 (local address 127.0.0.1:36881)
0811 07:11:08.343468 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:08.343478 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:08.343515 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:11:08.344431 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:08.344440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:08.344509 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:08.344673 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:08.344851 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:08.346284 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:08.347398 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:08.348048 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:08.348159 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:08.348961 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:08.348965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:08.348966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:08.349181 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:08.349221 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:08.349237 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:08.349307 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:08.349538 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 07:11:08.349610 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":103,"threads_started":1}
I20260811 07:11:08.350929 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:08.351744 7386 catalog_manager.cc:1347] Generated new cluster ID: a60e4c11e53046d1b5d34dd926643dab
I20260811 07:11:08.351799 7386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:08.355962 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:08.352504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58299 (local address 127.0.0.1:36881)
0811 07:11:08.352543 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:08.352545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:08.352676 (+ 131us) server_negotiation.cc:407] Connection header received
0811 07:11:08.352706 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:08.352707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:08.352718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:08.352782 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:08.352918 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:08.353653 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:08.354642 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:08.354946 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:08.355021 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:08.355719 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:08.355720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:08.355721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:08.355726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:08.355743 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:08.355748 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:08.355806 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:08.355865 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 07:11:08.355886 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:08.374095 7386 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:08.374661 7386 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:08.382118 7386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2c5ffd4dad94f0b9d4f2d8be7f62adc: Generated new TSK 0
I20260811 07:11:08.382344 7386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:08.391909 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7234
2026-08-11T07:11:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:08 dist-test-slave-2d19 krb5kdc[7421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:08 dist-test-slave-2d19 krb5kdc[7421](info): set up 2 sockets
Aug 11 07:11:08 dist-test-slave-2d19 krb5kdc[7421](info): commencing operation
krb5kdc: starting...
W20260811 07:11:09.683730 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:09 dist-test-slave-2d19 krb5kdc[7421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:09.810770 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:09.894224 7442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:09.894454 7442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:09.894493 7442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:09.895613 7442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:09.895687 7442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:09.895710 7442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:09.895730 7442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:09.895761 7442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:09.897240 7442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:09.897518 7442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:09.897753 7442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:09.902595 7451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:09.902606 7453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:09.902608 7458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:09.902711 7442 server_base.cc:1034] running on GCE node
I20260811 07:11:09.903059 7442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:09.903362 7442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:09.904524 7442 hybrid_clock.cc:648] HybridClock initialized: now 1786432269904502 us; error 39 us; skew 500 ppm
Aug 11 07:11:09 dist-test-slave-2d19 krb5kdc[7421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:09.906980 7442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:09.907461 7442 webserver.cc:460] Webserver started at http://127.0.0.1:40345/ using document root <none> and password file <none>
I20260811 07:11:09.907720 7442 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:09.907790 7442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:09.907936 7442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:09.909036 7442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "ee7001f8d9004f2fbf107f6e33f669a5"
format_stamp: "Formatted at 2026-08-11 07:11:09 on dist-test-slave-2d19"
server_key: "b757c67788bc474764faf03402d3ebcc"
server_key_iv: "ec1d225c41ad95ee9d6b0017c0d0f895"
server_key_version: "encryptionkey@0"
I20260811 07:11:09.909401 7442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "ee7001f8d9004f2fbf107f6e33f669a5"
format_stamp: "Formatted at 2026-08-11 07:11:09 on dist-test-slave-2d19"
server_key: "b757c67788bc474764faf03402d3ebcc"
server_key_iv: "ec1d225c41ad95ee9d6b0017c0d0f895"
server_key_version: "encryptionkey@0"
I20260811 07:11:09.910720 7442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 07:11:09.911463 7469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:09.911659 7442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:09.911751 7442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "ee7001f8d9004f2fbf107f6e33f669a5"
format_stamp: "Formatted at 2026-08-11 07:11:09 on dist-test-slave-2d19"
server_key: "b757c67788bc474764faf03402d3ebcc"
server_key_iv: "ec1d225c41ad95ee9d6b0017c0d0f895"
server_key_version: "encryptionkey@0"
I20260811 07:11:09.911832 7442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:09.915205 7442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:09.917112 7442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:09.917263 7442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:09.921562 7442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41385
I20260811 07:11:09.921595 7593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41385 every 8 connection(s)
I20260811 07:11:09.921965 7442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:09.923002 7594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:09.925312 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: Bootstrap starting.
I20260811 07:11:09.925884 7594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:09.926054 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7442
I20260811 07:11:09.926190 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:09.926239 7594 log.cc:824] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:09.926400 4018 external_mini_cluster.cc:1428] Setting key 9d7dec5da2966d6d4ed0da1e28f9c1e6
I20260811 07:11:09.927115 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: No bootstrap required, opened a new log
I20260811 07:11:09.928877 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:09.926946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60255 (local address 127.0.0.1:41385)
0811 07:11:09.927115 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:09.927123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:09.927205 (+ 82us) server_negotiation.cc:407] Connection header received
0811 07:11:09.927619 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:09.927625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:09.927649 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:09.927767 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:09.928422 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:09.928425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:09.928426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:09.928429 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:09.928465 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:09.928475 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:09.928543 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:09.928621 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 07:11:09.928672 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260811 07:11:09.929883 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:09.929833 7594 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260811 07:11:09.930024 7594 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:09.930063 7594 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7001f8d9004f2fbf107f6e33f669a5, State: Initialized, Role: FOLLOWER
I20260811 07:11:09.930172 7594 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260811 07:11:09.930253 7594 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:09.930281 7594 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:09.930330 7594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:09.931051 7594 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260811 07:11:09.931178 7594 leader_election.cc:302] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee7001f8d9004f2fbf107f6e33f669a5; no voters:
I20260811 07:11:09.931350 7594 leader_election.cc:288] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:09.931376 7611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:09.931510 7611 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [term 1 LEADER]: Becoming Leader. State: Replica: ee7001f8d9004f2fbf107f6e33f669a5, State: Running, Role: LEADER
I20260811 07:11:09.931622 7594 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:09.931668 7611 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260811 07:11:09.932034 7611 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7001f8d9004f2fbf107f6e33f669a5. Latest consensus state: current_term: 1 leader_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260811 07:11:09.932056 7612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7001f8d9004f2fbf107f6e33f669a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260811 07:11:09.932192 7612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:09.932452 7611 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:09.933522 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:09.931442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42635 (local address 127.0.0.1:41385)
0811 07:11:09.931479 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:09.931481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:09.932443 (+ 962us) server_negotiation.cc:407] Connection header received
0811 07:11:09.932561 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:09.932562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:09.932567 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:09.932596 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:09.933231 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:09.933233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:09.933233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:09.933238 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:09.933252 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:09.933256 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:09.933304 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:09.933443 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 07:11:09.933455 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 07:11:09.933682 7640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:09.933743 7640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:09.933837 7624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:09.934038 7624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:09.935539 7624 catalog_manager.cc:1347] Generated new cluster ID: 6459e3461f9941f9b65315409b931f8b
I20260811 07:11:09.935577 7624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:09.941510 7624 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:09.941660 7624 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:09.954162 7624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee7001f8d9004f2fbf107f6e33f669a5: Generated new TSK 0
I20260811 07:11:09.954394 7624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:09.963842 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7442
2026-08-11T07:11:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:10 dist-test-slave-2d19 krb5kdc[7649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:10 dist-test-slave-2d19 krb5kdc[7649](info): set up 2 sockets
Aug 11 07:11:10 dist-test-slave-2d19 krb5kdc[7649](info): commencing operation
krb5kdc: starting...
W20260811 07:11:11.251924 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:11.378934 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:11.463255 7669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:11.463521 7669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:11.463594 7669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:11.464864 7669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:11.464932 7669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:11.464954 7669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:11.464974 7669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:11.465003 7669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:11.466423 7669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:11.466722 7669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:11.466959 7669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:11.471868 7678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:11.471946 7677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:11.471901 7684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:11.472054 7669 server_base.cc:1034] running on GCE node
I20260811 07:11:11.472424 7669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:11.472740 7669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:11.473891 7669 hybrid_clock.cc:648] HybridClock initialized: now 1786432271473878 us; error 31 us; skew 500 ppm
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:11.476457 7669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:11.476944 7669 webserver.cc:460] Webserver started at http://127.0.0.1:35225/ using document root <none> and password file <none>
I20260811 07:11:11.477182 7669 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:11.477250 7669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:11.477409 7669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:11.478530 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "784fb21006fc4bf28bbb7341660008d4"
format_stamp: "Formatted at 2026-08-11 07:11:11 on dist-test-slave-2d19"
server_key: "f9218db5090e9606e326e33f0cf2fcb7"
server_key_iv: "4ce6c1082e9988fd4c580d741baf7b88"
server_key_version: "encryptionkey@0"
I20260811 07:11:11.478926 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "784fb21006fc4bf28bbb7341660008d4"
format_stamp: "Formatted at 2026-08-11 07:11:11 on dist-test-slave-2d19"
server_key: "f9218db5090e9606e326e33f0cf2fcb7"
server_key_iv: "4ce6c1082e9988fd4c580d741baf7b88"
server_key_version: "encryptionkey@0"
I20260811 07:11:11.480365 7669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:11.481135 7698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:11.481299 7669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:11.481411 7669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "784fb21006fc4bf28bbb7341660008d4"
format_stamp: "Formatted at 2026-08-11 07:11:11 on dist-test-slave-2d19"
server_key: "f9218db5090e9606e326e33f0cf2fcb7"
server_key_iv: "4ce6c1082e9988fd4c580d741baf7b88"
server_key_version: "encryptionkey@0"
I20260811 07:11:11.481487 7669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:11.506788 7669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:11.509336 7669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:11.509526 7669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:11.513849 7669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46323
I20260811 07:11:11.513890 7802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46323 every 8 connection(s)
I20260811 07:11:11.514281 7669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:11.514899 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7669
I20260811 07:11:11.515031 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:11.515231 4018 external_mini_cluster.cc:1428] Setting key d30ba79f2324bc2cc90cc91526d8d69d
I20260811 07:11:11.515414 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:11.518885 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4: Bootstrap starting.
I20260811 07:11:11.519619 7804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:11.519917 7804 log.cc:824] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:11.520655 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4: No bootstrap required, opened a new log
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:11:11.522928 7804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 07:11:11.523097 7804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:11.523152 7804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784fb21006fc4bf28bbb7341660008d4, State: Initialized, Role: FOLLOWER
I20260811 07:11:11.523288 7804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 07:11:11.523409 7804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:11.523460 7804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:11.523537 7804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:11.524209 7804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 07:11:11.524354 7804 leader_election.cc:302] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 784fb21006fc4bf28bbb7341660008d4; no voters:
I20260811 07:11:11.524538 7804 leader_election.cc:288] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:11.524582 7815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:11.524766 7815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [term 1 LEADER]: Becoming Leader. State: Replica: 784fb21006fc4bf28bbb7341660008d4, State: Running, Role: LEADER
I20260811 07:11:11.524811 7804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:11.524889 7815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } }
I20260811 07:11:11.525460 7816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "784fb21006fc4bf28bbb7341660008d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } } }
I20260811 07:11:11.525475 7815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 784fb21006fc4bf28bbb7341660008d4. Latest consensus state: current_term: 1 leader_uuid: "784fb21006fc4bf28bbb7341660008d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784fb21006fc4bf28bbb7341660008d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46323 } } }
I20260811 07:11:11.525570 7816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:11.525578 7815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:11.525861 7833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:11.526993 7833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:11.527381 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:11.515900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:52229 (local address 127.0.0.1:46323)
0811 07:11:11.516101 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:11.516110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:11.516145 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:11:11.516631 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:11.516638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:11.516689 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:11.516805 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:11.517597 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:11.518485 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:11.519525 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:11.519979 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:11.520140 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:11.522835 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:11.522839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:11.522839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:11.522849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:11.524872 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:11.525405 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:11.525408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:11.525409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:11.525555 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:11.525860 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:11.525860 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:11.525861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:11.526219 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:11.526793 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:11.526881 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 07:11:11.526984 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260811 07:11:11.528388 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:11.528604 7833 catalog_manager.cc:1347] Generated new cluster ID: 401ce8339c2f4c2ab19243d9cf0eb42d
I20260811 07:11:11.528652 7833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:11.537180 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:11.529965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:47999 (local address 127.0.0.1:46323)
0811 07:11:11.530014 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:11.530016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:11.530023 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:11:11.530127 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:11.530128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:11.530139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:11.530204 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:11.530805 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:11.531627 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:11.532651 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:11.532975 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:11.533052 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:11.534330 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:11.534332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:11.534332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:11.534339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:11.535949 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:11.536366 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:11.536367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:11.536367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:11.536427 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:11.536669 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:11.536671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:11.536671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:11.536800 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:11.536909 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:11.537023 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 07:11:11.537089 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:11:11.553395 7833 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:11.553990 7833 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:11.566545 7833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 784fb21006fc4bf28bbb7341660008d4: Generated new TSK 0
I20260811 07:11:11.566767 7833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:11.571761 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7669
W20260811 07:11:11.592707 7643 connection.cc:535] client connection to 127.0.0.1:46323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46323): Connection reset by peer (error 104)
2026-08-11T07:11:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7856](info): set up 2 sockets
Aug 11 07:11:11 dist-test-slave-2d19 krb5kdc[7856](info): commencing operation
krb5kdc: starting...
W20260811 07:11:12.874792 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:12 dist-test-slave-2d19 krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:13.003057 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:13.086460 7885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:13.086674 7885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:13.086696 7885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:13.087847 7885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:13.087893 7885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:13.087908 7885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:13.087921 7885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:13.087934 7885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:13.089480 7885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:13.089710 7885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:13.089923 7885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:13.094779 7907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:13.094792 7914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:13.094869 7908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:13.095093 7885 server_base.cc:1034] running on GCE node
I20260811 07:11:13.095297 7885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:13.095530 7885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:13.096694 7885 hybrid_clock.cc:648] HybridClock initialized: now 1786432273096689 us; error 38 us; skew 500 ppm
Aug 11 07:11:13 dist-test-slave-2d19 krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:13.099231 7885 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:13.099668 7885 webserver.cc:460] Webserver started at http://127.0.0.1:41437/ using document root <none> and password file <none>
I20260811 07:11:13.099913 7885 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:13.099984 7885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:13.100137 7885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:13.101235 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "7eabcdb3a3ea44b5b20a144255a159e8"
format_stamp: "Formatted at 2026-08-11 07:11:13 on dist-test-slave-2d19"
server_key: "90c95a48c18d1149643fe9a87ec5bbb8"
server_key_iv: "6e59cae16383a95758384c95b7716632"
server_key_version: "encryptionkey@0"
I20260811 07:11:13.101608 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "7eabcdb3a3ea44b5b20a144255a159e8"
format_stamp: "Formatted at 2026-08-11 07:11:13 on dist-test-slave-2d19"
server_key: "90c95a48c18d1149643fe9a87ec5bbb8"
server_key_iv: "6e59cae16383a95758384c95b7716632"
server_key_version: "encryptionkey@0"
I20260811 07:11:13.103080 7885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:11:13.103852 7924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:13.104034 7885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:13.104139 7885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "7eabcdb3a3ea44b5b20a144255a159e8"
format_stamp: "Formatted at 2026-08-11 07:11:13 on dist-test-slave-2d19"
server_key: "90c95a48c18d1149643fe9a87ec5bbb8"
server_key_iv: "6e59cae16383a95758384c95b7716632"
server_key_version: "encryptionkey@0"
I20260811 07:11:13.104220 7885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:13.115551 7885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:13.117440 7885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:13.117605 7885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:13.121727 7885 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45523
I20260811 07:11:13.121775 8016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45523 every 8 connection(s)
I20260811 07:11:13.122138 7885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:13.123231 8017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:13.125633 8017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8: Bootstrap starting.
I20260811 07:11:13.126235 8017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:13.126544 8017 log.cc:824] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:13.127277 8017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8: No bootstrap required, opened a new log
I20260811 07:11:13.127995 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 7885
I20260811 07:11:13.128149 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:13.128384 4018 external_mini_cluster.cc:1428] Setting key bae37062eba73b634e15c38254ef9192
I20260811 07:11:13.129451 8017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } }
I20260811 07:11:13.129648 8017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:13.129716 8017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabcdb3a3ea44b5b20a144255a159e8, State: Initialized, Role: FOLLOWER
I20260811 07:11:13.129911 8017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } }
I20260811 07:11:13.130014 8017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:13.130062 8017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:13.130117 8017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:13.130851 8017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } }
I20260811 07:11:13.130962 8017 leader_election.cc:302] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eabcdb3a3ea44b5b20a144255a159e8; no voters:
I20260811 07:11:13.131117 8017 leader_election.cc:288] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:13.131175 8023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:13.131356 8017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:13.131358 8023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [term 1 LEADER]: Becoming Leader. State: Replica: 7eabcdb3a3ea44b5b20a144255a159e8, State: Running, Role: LEADER
I20260811 07:11:13.131501 8023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } }
I20260811 07:11:13.131889 8023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eabcdb3a3ea44b5b20a144255a159e8. Latest consensus state: current_term: 1 leader_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } } }
I20260811 07:11:13.131974 8023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:13.132148 8024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eabcdb3a3ea44b5b20a144255a159e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45523 } } }
I20260811 07:11:13.132229 8024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:13.132380 8047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:13.133237 8047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:13.135107 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:13.128980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:51785 (local address 127.0.0.1:45523)
0811 07:11:13.129172 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:13.129180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:13.129207 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:11:13.129405 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:13.129410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:13.129456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:13.129565 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:13.129844 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:13.131596 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:13.132822 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:13.133455 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:13.133938 (+ 483us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:13.134359 (+ 421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:13.134363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:13.134363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:13.134605 (+ 242us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:13.134647 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:13.134661 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:13.134779 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:13.134862 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 07:11:13.134894 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260811 07:11:13.136112 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:13.136639 8047 catalog_manager.cc:1347] Generated new cluster ID: 0d79014ce3b14468be8db30c13c69190
I20260811 07:11:13.136690 8047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:13.144090 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:13.140535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:57371 (local address 127.0.0.1:45523)
0811 07:11:13.140568 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:13.140570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:13.140730 (+ 160us) server_negotiation.cc:407] Connection header received
0811 07:11:13.140757 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:13.140758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:13.140770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:13.140839 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:13.140975 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:13.141689 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:13.142720 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:13.143058 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:13.143139 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:13.143837 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:13.143838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:13.143838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:13.143843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:13.143858 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:13.143862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:13.143927 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:13.144001 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 07:11:13.144015 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 07:11:13.149801 8047 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:13.150480 8047 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:13.162645 8047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eabcdb3a3ea44b5b20a144255a159e8: Generated new TSK 0
I20260811 07:11:13.162865 8047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:13.167816 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 7885
W20260811 07:11:13.190271 7854 connection.cc:535] client connection to 127.0.0.1:45523 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45523): Connection reset by peer (error 104)
2026-08-11T07:11:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:13 dist-test-slave-2d19 krb5kdc[8074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:13 dist-test-slave-2d19 krb5kdc[8074](info): set up 2 sockets
Aug 11 07:11:13 dist-test-slave-2d19 krb5kdc[8074](info): commencing operation
krb5kdc: starting...
W20260811 07:11:14.475020 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:14 dist-test-slave-2d19 krb5kdc[8074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:14.603562 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:14.687282 8098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:14.687487 8098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:14.687510 8098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:14.688612 8098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:14.688647 8098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:14.688663 8098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:14.688674 8098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:14.688686 8098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:14.690084 8098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:14.690277 8098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:14.690449 8098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:14.695511 8115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:14.695519 8110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:14.695525 8109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:14.695562 8098 server_base.cc:1034] running on GCE node
I20260811 07:11:14.695986 8098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:14.696346 8098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:14.697500 8098 hybrid_clock.cc:648] HybridClock initialized: now 1786432274697488 us; error 31 us; skew 500 ppm
Aug 11 07:11:14 dist-test-slave-2d19 krb5kdc[8074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432274, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:14.700037 8098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:14.700542 8098 webserver.cc:460] Webserver started at http://127.0.0.1:36007/ using document root <none> and password file <none>
I20260811 07:11:14.700794 8098 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:14.700840 8098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:14.701013 8098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:14.702071 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "666c13c64c2e4d0f94e73e46427bfad4"
format_stamp: "Formatted at 2026-08-11 07:11:14 on dist-test-slave-2d19"
server_key: "5f42fce3a7dcf59048ecdbb57ebcdb83"
server_key_iv: "32f9a82499542e864879f469ae90ae0d"
server_key_version: "encryptionkey@0"
I20260811 07:11:14.702454 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "666c13c64c2e4d0f94e73e46427bfad4"
format_stamp: "Formatted at 2026-08-11 07:11:14 on dist-test-slave-2d19"
server_key: "5f42fce3a7dcf59048ecdbb57ebcdb83"
server_key_iv: "32f9a82499542e864879f469ae90ae0d"
server_key_version: "encryptionkey@0"
I20260811 07:11:14.703874 8098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:14.704696 8126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:14.704893 8098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:11:14.705013 8098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "666c13c64c2e4d0f94e73e46427bfad4"
format_stamp: "Formatted at 2026-08-11 07:11:14 on dist-test-slave-2d19"
server_key: "5f42fce3a7dcf59048ecdbb57ebcdb83"
server_key_iv: "32f9a82499542e864879f469ae90ae0d"
server_key_version: "encryptionkey@0"
I20260811 07:11:14.705091 8098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:14.708366 8098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:14.710687 8098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:14.710839 8098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:14.715052 8098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39589
I20260811 07:11:14.715088 8229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39589 every 8 connection(s)
I20260811 07:11:14.715451 8098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:14.716506 8230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:14.718834 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4: Bootstrap starting.
I20260811 07:11:14.718902 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8098
I20260811 07:11:14.719007 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:14.719223 4018 external_mini_cluster.cc:1428] Setting key 7568d6c98df6dfba62c6f19f5496f1a9
I20260811 07:11:14.719585 8230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:14.719937 8230 log.cc:824] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:14.720898 8230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4: No bootstrap required, opened a new log
I20260811 07:11:14.721807 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:14.719736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:43269 (local address 127.0.0.1:39589)
0811 07:11:14.719931 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:14.719938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:14.719973 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:11:14.720528 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:14.720535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:14.720562 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:14.720681 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:14.721347 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:14.721350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:14.721350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:14.721354 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:14.721393 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:14.721403 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:14.721479 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:14.721552 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 07:11:14.721608 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260811 07:11:14.722846 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:14.723557 8230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } }
I20260811 07:11:14.723742 8230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:14.723778 8230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666c13c64c2e4d0f94e73e46427bfad4, State: Initialized, Role: FOLLOWER
I20260811 07:11:14.723891 8230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } }
I20260811 07:11:14.723996 8230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:14.724059 8230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:14.724121 8230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:14.724908 8230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } }
I20260811 07:11:14.725051 8230 leader_election.cc:302] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 666c13c64c2e4d0f94e73e46427bfad4; no voters:
I20260811 07:11:14.725219 8230 leader_election.cc:288] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:14.725277 8255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:14.725433 8255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [term 1 LEADER]: Becoming Leader. State: Replica: 666c13c64c2e4d0f94e73e46427bfad4, State: Running, Role: LEADER
I20260811 07:11:14.725487 8230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:14.725531 8255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } }
I20260811 07:11:14.725602 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:14.724518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:59589 (local address 127.0.0.1:39589)
0811 07:11:14.724556 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:14.724557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:14.724625 (+ 68us) server_negotiation.cc:407] Connection header received
0811 07:11:14.724693 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:14.724694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:14.724697 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:14.724759 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:14.725359 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:14.725360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:14.725361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:14.725365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:14.725379 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:14.725383 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:14.725412 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:14.725536 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 07:11:14.725551 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:11:14.725847 8255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666c13c64c2e4d0f94e73e46427bfad4. Latest consensus state: current_term: 1 leader_uuid: "666c13c64c2e4d0f94e73e46427bfad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } } }
I20260811 07:11:14.725950 8255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:14.725857 8258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "666c13c64c2e4d0f94e73e46427bfad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666c13c64c2e4d0f94e73e46427bfad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39589 } } }
I20260811 07:11:14.726045 8258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:14.726881 8263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:14.727478 8263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:14.728992 8263 catalog_manager.cc:1347] Generated new cluster ID: 086f02e98a6547d2a5c0526992bb05c2
I20260811 07:11:14.729068 8263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:14.748421 8263 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:14.748687 8263 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:14.757140 8263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 666c13c64c2e4d0f94e73e46427bfad4: Generated new TSK 0
I20260811 07:11:14.757369 8263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:14.769690 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8098
2026-08-11T07:11:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:14 dist-test-slave-2d19 krb5kdc[8292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:14 dist-test-slave-2d19 krb5kdc[8292](info): set up 2 sockets
Aug 11 07:11:14 dist-test-slave-2d19 krb5kdc[8292](info): commencing operation
krb5kdc: starting...
W20260811 07:11:16.103578 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:16.229465 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:16.313432 8315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:16.313638 8315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:16.313661 8315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:16.314728 8315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:16.314761 8315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:16.314775 8315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:16.314788 8315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:16.314800 8315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:16.316210 8315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:16.316460 8315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:16.316648 8315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:16.321574 8324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:16.321640 8329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:16.321609 8315 server_base.cc:1034] running on GCE node
W20260811 07:11:16.321574 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:16.322082 8315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:16.322404 8315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:16.323582 8315 hybrid_clock.cc:648] HybridClock initialized: now 1786432276323579 us; error 35 us; skew 500 ppm
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:16.326128 8315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:16.326570 8315 webserver.cc:460] Webserver started at http://127.0.0.1:36589/ using document root <none> and password file <none>
I20260811 07:11:16.326826 8315 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:16.326897 8315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:16.327066 8315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:16.328478 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "d8091ec6496d4bb3a5e0495e9456405c"
format_stamp: "Formatted at 2026-08-11 07:11:16 on dist-test-slave-2d19"
server_key: "43a6a72c48159044adbb3fbfc99433c1"
server_key_iv: "2281e316222dc6f7c356883aa364f413"
server_key_version: "encryptionkey@0"
I20260811 07:11:16.328871 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "d8091ec6496d4bb3a5e0495e9456405c"
format_stamp: "Formatted at 2026-08-11 07:11:16 on dist-test-slave-2d19"
server_key: "43a6a72c48159044adbb3fbfc99433c1"
server_key_iv: "2281e316222dc6f7c356883aa364f413"
server_key_version: "encryptionkey@0"
I20260811 07:11:16.330235 8315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:16.331004 8345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:16.331199 8315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:16.331298 8315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "d8091ec6496d4bb3a5e0495e9456405c"
format_stamp: "Formatted at 2026-08-11 07:11:16 on dist-test-slave-2d19"
server_key: "43a6a72c48159044adbb3fbfc99433c1"
server_key_iv: "2281e316222dc6f7c356883aa364f413"
server_key_version: "encryptionkey@0"
I20260811 07:11:16.331375 8315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:16.344930 8315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:16.347242 8315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:16.347447 8315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:16.351701 8315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43897
I20260811 07:11:16.351754 8436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43897 every 8 connection(s)
I20260811 07:11:16.352145 8315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:16.353305 8437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:16.354887 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8315
I20260811 07:11:16.354975 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:16.355176 4018 external_mini_cluster.cc:1428] Setting key 698c8d06623fba6e87911595e3be19eb
I20260811 07:11:16.356128 8437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c: Bootstrap starting.
I20260811 07:11:16.356911 8437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:16.357239 8437 log.cc:824] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:16.358045 8437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c: No bootstrap required, opened a new log
I20260811 07:11:16.360394 8437 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } }
I20260811 07:11:16.360618 8437 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:16.360698 8437 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8091ec6496d4bb3a5e0495e9456405c, State: Initialized, Role: FOLLOWER
I20260811 07:11:16.360844 8437 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } }
I20260811 07:11:16.360957 8437 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:16.361008 8437 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:16.361070 8437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:16.361931 8437 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } }
I20260811 07:11:16.362082 8437 leader_election.cc:302] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8091ec6496d4bb3a5e0495e9456405c; no voters:
I20260811 07:11:16.362298 8437 leader_election.cc:288] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:16.362352 8450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:16.362540 8450 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [term 1 LEADER]: Becoming Leader. State: Replica: d8091ec6496d4bb3a5e0495e9456405c, State: Running, Role: LEADER
I20260811 07:11:16.362612 8437 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:16.362697 8450 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } }
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:11:16.363094 8450 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8091ec6496d4bb3a5e0495e9456405c. Latest consensus state: current_term: 1 leader_uuid: "d8091ec6496d4bb3a5e0495e9456405c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } } }
I20260811 07:11:16.363101 8454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8091ec6496d4bb3a5e0495e9456405c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8091ec6496d4bb3a5e0495e9456405c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43897 } } }
I20260811 07:11:16.363296 8450 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:16.363317 8454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:16.363694 8465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:16.364965 8465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:16.366909 8465 catalog_manager.cc:1347] Generated new cluster ID: 28ddf674487945bf9bc3aa175cbabb1c
I20260811 07:11:16.366964 8465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:16.367764 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:16.355757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:58255 (local address 127.0.0.1:43897)
0811 07:11:16.355974 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:16.355983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:16.356019 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:11:16.356647 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:16.356658 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:16.356725 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:16.356887 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:16.357583 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:16.358643 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:16.359756 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:16.360176 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:16.360311 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:16.363567 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:16.363571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:16.363572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:16.363584 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:16.365803 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:16.366335 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:16.366339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:16.366339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:16.366456 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:16.366766 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:16.366767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:16.366768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:16.367097 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:16.367266 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:16.367458 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 07:11:16.367561 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260811 07:11:16.368902 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:16.373648 8465 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:16.374341 8465 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:16.378057 8465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8091ec6496d4bb3a5e0495e9456405c: Generated new TSK 0
I20260811 07:11:16.378243 8465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:16.378932 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:16.370378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:49665 (local address 127.0.0.1:43897)
0811 07:11:16.370423 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:16.370425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:16.370489 (+ 64us) server_negotiation.cc:407] Connection header received
0811 07:11:16.370560 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:16.370561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:16.370571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:16.370626 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:16.371343 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:16.372491 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:16.373542 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:16.373841 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:16.373915 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:16.375614 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:16.375616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:16.375616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:16.375624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:16.377452 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:16.377966 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:16.377968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:16.377968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:16.378038 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:16.378464 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:16.378465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:16.378466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:16.378572 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:16.378657 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:16.378820 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:11:16.378872 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:11:16.381345 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8315
W20260811 07:11:16.407433 8290 connection.cc:535] client connection to 127.0.0.1:43897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43897): Connection reset by peer (error 104)
2026-08-11T07:11:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8492](info): set up 2 sockets
Aug 11 07:11:16 dist-test-slave-2d19 krb5kdc[8492](info): commencing operation
krb5kdc: starting...
W20260811 07:11:17.683977 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:17 dist-test-slave-2d19 krb5kdc[8492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:17.810662 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:17.895058 8513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:17.895370 8513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:17.895412 8513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:17.896597 8513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:17.896669 8513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:17.896693 8513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:17.896715 8513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:17.896742 8513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:17.898263 8513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:17.898561 8513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:17.898802 8513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:17.904366 8523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:17.904538 8513 server_base.cc:1034] running on GCE node
W20260811 07:11:17.904652 8527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:17.904680 8522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:17.904922 8513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:17.905233 8513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:17.906401 8513 hybrid_clock.cc:648] HybridClock initialized: now 1786432277906385 us; error 32 us; skew 500 ppm
Aug 11 07:11:17 dist-test-slave-2d19 krb5kdc[8492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:17.909010 8513 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:17.909503 8513 webserver.cc:460] Webserver started at http://127.0.0.1:37015/ using document root <none> and password file <none>
I20260811 07:11:17.909746 8513 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:17.909792 8513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:17.909968 8513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:17.911068 8513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "2c1baee7ca514548aeb75ed3e45c9949"
format_stamp: "Formatted at 2026-08-11 07:11:17 on dist-test-slave-2d19"
server_key: "37a7c64e690fd819099a4c87a895d252"
server_key_iv: "b06c611e0392d98313a2346608be730c"
server_key_version: "encryptionkey@0"
I20260811 07:11:17.911443 8513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "2c1baee7ca514548aeb75ed3e45c9949"
format_stamp: "Formatted at 2026-08-11 07:11:17 on dist-test-slave-2d19"
server_key: "37a7c64e690fd819099a4c87a895d252"
server_key_iv: "b06c611e0392d98313a2346608be730c"
server_key_version: "encryptionkey@0"
I20260811 07:11:17.912889 8513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:17.913727 8543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:17.913923 8513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:17.914048 8513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "2c1baee7ca514548aeb75ed3e45c9949"
format_stamp: "Formatted at 2026-08-11 07:11:17 on dist-test-slave-2d19"
server_key: "37a7c64e690fd819099a4c87a895d252"
server_key_iv: "b06c611e0392d98313a2346608be730c"
server_key_version: "encryptionkey@0"
I20260811 07:11:17.914125 8513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:17.926172 8513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:17.928025 8513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:17.928186 8513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:17.932425 8513 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39905
I20260811 07:11:17.932469 8641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39905 every 8 connection(s)
I20260811 07:11:17.932833 8513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:17.933713 8644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:17.935936 8644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949: Bootstrap starting.
I20260811 07:11:17.936569 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8513
I20260811 07:11:17.936580 8644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:17.936714 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:17.936919 4018 external_mini_cluster.cc:1428] Setting key 1d8dec644325f23323b066ad82bff878
I20260811 07:11:17.936969 8644 log.cc:824] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:17.937832 8644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949: No bootstrap required, opened a new log
I20260811 07:11:17.940233 8644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } }
I20260811 07:11:17.940433 8644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:17.940464 8644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1baee7ca514548aeb75ed3e45c9949, State: Initialized, Role: FOLLOWER
I20260811 07:11:17.940589 8644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } }
I20260811 07:11:17.940665 8644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:17.940701 8644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:17.940784 8644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:17.941484 8644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } }
I20260811 07:11:17.941597 8644 leader_election.cc:302] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c1baee7ca514548aeb75ed3e45c9949; no voters:
I20260811 07:11:17.941781 8644 leader_election.cc:288] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:17.941803 8651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:17.941921 8651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [term 1 LEADER]: Becoming Leader. State: Replica: 2c1baee7ca514548aeb75ed3e45c9949, State: Running, Role: LEADER
I20260811 07:11:17.942051 8644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:17.942024 8651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } }
I20260811 07:11:17.942407 8653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c1baee7ca514548aeb75ed3e45c9949. Latest consensus state: current_term: 1 leader_uuid: "2c1baee7ca514548aeb75ed3e45c9949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } } }
I20260811 07:11:17.942667 8653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:17.942584 8654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c1baee7ca514548aeb75ed3e45c9949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1baee7ca514548aeb75ed3e45c9949" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39905 } } }
I20260811 07:11:17.942936 8654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:17.943161 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:17.937511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:46217 (local address 127.0.0.1:39905)
0811 07:11:17.937674 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:17.937683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:17.937720 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:11:17.938229 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:17.938237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:17.938288 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:17.938408 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:17.938667 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:17.939520 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:17.940779 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:17.941396 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:17.941517 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:17.942302 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:17.942306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:17.942307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:17.942543 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:17.942585 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:17.942601 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:17.942689 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:17.942889 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 07:11:17.942920 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260811 07:11:17.943173 8668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:17.943913 8668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:17.944155 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:17.948992 8668 catalog_manager.cc:1347] Generated new cluster ID: 7f31f719dc5b4e4c9edb9b608217f730
I20260811 07:11:17.949054 8668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:17.949150 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:17.945722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:55835 (local address 127.0.0.1:39905)
0811 07:11:17.945760 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:17.945761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:17.945838 (+ 77us) server_negotiation.cc:407] Connection header received
0811 07:11:17.945907 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:17.945907 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:17.945916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:17.945984 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:17.946137 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:17.946581 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:17.947646 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:17.947875 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:17.947940 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:17.948714 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:17.948716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:17.948716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:17.948720 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:17.948732 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:17.948736 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:17.948846 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:17.949029 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 07:11:17.949042 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:17.958621 8668 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:17.959178 8668 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:17.962643 8668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c1baee7ca514548aeb75ed3e45c9949: Generated new TSK 0
I20260811 07:11:17.962831 8668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:17.969635 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8513
W20260811 07:11:17.975217 8489 connection.cc:535] client connection to 127.0.0.1:39905 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39905): Connection reset by peer (error 104)
2026-08-11T07:11:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:18 dist-test-slave-2d19 krb5kdc[8696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:18 dist-test-slave-2d19 krb5kdc[8696](info): set up 2 sockets
Aug 11 07:11:18 dist-test-slave-2d19 krb5kdc[8696](info): commencing operation
krb5kdc: starting...
W20260811 07:11:19.253496 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:19 dist-test-slave-2d19 krb5kdc[8696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:11:19.378906 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:19.463425 8718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:19.463687 8718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:19.463725 8718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:19.465000 8718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:19.465067 8718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:19.465091 8718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:19.465112 8718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:19.465142 8718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:19.466636 8718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:19.466925 8718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:19.467167 8718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:19.472139 8730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:19.472204 8732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:19.472131 8729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:19.472468 8718 server_base.cc:1034] running on GCE node
I20260811 07:11:19.472736 8718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:19.473027 8718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:19.474200 8718 hybrid_clock.cc:648] HybridClock initialized: now 1786432279474166 us; error 54 us; skew 500 ppm
Aug 11 07:11:19 dist-test-slave-2d19 krb5kdc[8696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:19.482286 8718 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:11:19.482781 8718 webserver.cc:460] Webserver started at http://127.0.0.1:39341/ using document root <none> and password file <none>
I20260811 07:11:19.483045 8718 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:19.483117 8718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:19.483270 8718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:19.484470 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "54540d787f674c8dbc6ed6d0ed44a83c"
format_stamp: "Formatted at 2026-08-11 07:11:19 on dist-test-slave-2d19"
server_key: "795c57231ab5c824e0b8367b586803aa"
server_key_iv: "e3c63418147d9860c3a180384e08d1f3"
server_key_version: "encryptionkey@0"
I20260811 07:11:19.484898 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "54540d787f674c8dbc6ed6d0ed44a83c"
format_stamp: "Formatted at 2026-08-11 07:11:19 on dist-test-slave-2d19"
server_key: "795c57231ab5c824e0b8367b586803aa"
server_key_iv: "e3c63418147d9860c3a180384e08d1f3"
server_key_version: "encryptionkey@0"
I20260811 07:11:19.486277 8718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:19.487038 8742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:19.487231 8718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:19.487317 8718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "54540d787f674c8dbc6ed6d0ed44a83c"
format_stamp: "Formatted at 2026-08-11 07:11:19 on dist-test-slave-2d19"
server_key: "795c57231ab5c824e0b8367b586803aa"
server_key_iv: "e3c63418147d9860c3a180384e08d1f3"
server_key_version: "encryptionkey@0"
I20260811 07:11:19.487407 8718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:19.490926 8718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:19.492930 8718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:19.493085 8718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:19.497253 8718 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41295
I20260811 07:11:19.497327 8841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41295 every 8 connection(s)
I20260811 07:11:19.497642 8718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:19.498526 8843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:19.500898 8843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c: Bootstrap starting.
I20260811 07:11:19.501446 8843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:19.501698 8843 log.cc:824] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:19.502363 8843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c: No bootstrap required, opened a new log
I20260811 07:11:19.503955 8843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } }
I20260811 07:11:19.504096 8843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:19.504119 8843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54540d787f674c8dbc6ed6d0ed44a83c, State: Initialized, Role: FOLLOWER
I20260811 07:11:19.504259 8843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } }
I20260811 07:11:19.504370 8843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:19.504395 8843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:19.504426 8843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:19.504940 8843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } }
I20260811 07:11:19.504976 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8718
I20260811 07:11:19.505025 8843 leader_election.cc:302] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54540d787f674c8dbc6ed6d0ed44a83c; no voters:
I20260811 07:11:19.505155 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:19.505249 8843 leader_election.cc:288] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:19.505283 8850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:19.505381 4018 external_mini_cluster.cc:1428] Setting key 53767d09309fe20eca921c5172422980
I20260811 07:11:19.505445 8850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [term 1 LEADER]: Becoming Leader. State: Replica: 54540d787f674c8dbc6ed6d0ed44a83c, State: Running, Role: LEADER
I20260811 07:11:19.505533 8850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } }
I20260811 07:11:19.505532 8843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:19.506445 8858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54540d787f674c8dbc6ed6d0ed44a83c. Latest consensus state: current_term: 1 leader_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } } }
I20260811 07:11:19.506557 8858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:19.506778 8857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54540d787f674c8dbc6ed6d0ed44a83c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41295 } } }
I20260811 07:11:19.506914 8857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:19.507114 8875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:19.507655 8875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:19.508186 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:19.505958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:60061 (local address 127.0.0.1:41295)
0811 07:11:19.506164 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:19.506174 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:19.506208 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:11:19.506335 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:19.506346 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:19.506376 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:19.506527 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:19.507665 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:19.507668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:19.507669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:19.507672 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:19.507732 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:19.507747 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:19.507808 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:19.507932 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 07:11:19.507984 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260811 07:11:19.509209 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:19.509244 8875 catalog_manager.cc:1347] Generated new cluster ID: 23fee05045ac47398e7f6c3ceb14c089
I20260811 07:11:19.509279 8875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:19.511910 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:19.510725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:49601 (local address 127.0.0.1:41295)
0811 07:11:19.510758 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:19.510760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:19.510890 (+ 130us) server_negotiation.cc:407] Connection header received
0811 07:11:19.510954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:19.510955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:19.510959 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:19.511023 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:19.511648 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:19.511649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:19.511649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:19.511653 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:19.511667 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:19.511671 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:19.511724 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:19.511807 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 07:11:19.511819 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 07:11:19.516302 8875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:19.516472 8875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:19.523720 8875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54540d787f674c8dbc6ed6d0ed44a83c: Generated new TSK 0
I20260811 07:11:19.523954 8875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:19.526023 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8718
2026-08-11T07:11:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:19 dist-test-slave-2d19 krb5kdc[8900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:19 dist-test-slave-2d19 krb5kdc[8900](info): set up 2 sockets
Aug 11 07:11:19 dist-test-slave-2d19 krb5kdc[8900](info): commencing operation
krb5kdc: starting...
W20260811 07:11:20.803203 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:20 dist-test-slave-2d19 krb5kdc[8900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:20.930567 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42285
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:21.013623 8923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:21.013870 8923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:21.013909 8923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:21.015067 8923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:21.015141 8923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:21.015164 8923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:21.015187 8923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:21.015213 8923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:21.016667 8923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42285
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:21.016947 8923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:21.017184 8923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:21.022203 8935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:21.022152 8933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:21.022225 8923 server_base.cc:1034] running on GCE node
W20260811 07:11:21.022148 8938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:21.022689 8923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:21.023041 8923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:21.024209 8923 hybrid_clock.cc:648] HybridClock initialized: now 1786432281024204 us; error 40 us; skew 500 ppm
Aug 11 07:11:21 dist-test-slave-2d19 krb5kdc[8900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432281, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:21.026680 8923 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:21.027163 8923 webserver.cc:460] Webserver started at http://127.3.236.190:43443/ using document root <none> and password file <none>
I20260811 07:11:21.027413 8923 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:21.027484 8923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:21.027639 8923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:21.028772 8923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "e083ad897da442059331d3fdebede5fe"
format_stamp: "Formatted at 2026-08-11 07:11:21 on dist-test-slave-2d19"
server_key: "e526d8972bcf96188e7b165628909ae6"
server_key_iv: "a8f162724b62987bec91292810949f0f"
server_key_version: "encryptionkey@0"
I20260811 07:11:21.029151 8923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "e083ad897da442059331d3fdebede5fe"
format_stamp: "Formatted at 2026-08-11 07:11:21 on dist-test-slave-2d19"
server_key: "e526d8972bcf96188e7b165628909ae6"
server_key_iv: "a8f162724b62987bec91292810949f0f"
server_key_version: "encryptionkey@0"
I20260811 07:11:21.030483 8923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 07:11:21.031231 8948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:21.031409 8923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:21.031505 8923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "e083ad897da442059331d3fdebede5fe"
format_stamp: "Formatted at 2026-08-11 07:11:21 on dist-test-slave-2d19"
server_key: "e526d8972bcf96188e7b165628909ae6"
server_key_iv: "a8f162724b62987bec91292810949f0f"
server_key_version: "encryptionkey@0"
I20260811 07:11:21.031596 8923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:21.064853 8923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:21.066861 8923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:21.067049 8923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:21.071623 8923 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:42285
I20260811 07:11:21.071672 9042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:42285 every 8 connection(s)
I20260811 07:11:21.072098 8923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:21.073060 9045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:21.075419 9045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe: Bootstrap starting.
I20260811 07:11:21.075964 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 8923
I20260811 07:11:21.076027 9045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:21.076123 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:21.076336 4018 external_mini_cluster.cc:1428] Setting key cf0cf2bd01e5bc32a4513c7c02bab0cc
I20260811 07:11:21.076442 9045 log.cc:824] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:21.077293 9045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe: No bootstrap required, opened a new log
I20260811 07:11:21.078867 9045 raft_consensus.cc:348] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } }
I20260811 07:11:21.079016 9045 raft_consensus.cc:374] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:21.079041 9045 raft_consensus.cc:729] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e083ad897da442059331d3fdebede5fe, State: Initialized, Role: FOLLOWER
I20260811 07:11:21.079192 9045 consensus_queue.cc:260] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } }
I20260811 07:11:21.079259 9045 raft_consensus.cc:388] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:21.079310 9045 raft_consensus.cc:482] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:21.079367 9045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:21.079921 9045 raft_consensus.cc:504] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } }
I20260811 07:11:21.080011 9045 leader_election.cc:302] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e083ad897da442059331d3fdebede5fe; no voters:
I20260811 07:11:21.080214 9045 leader_election.cc:288] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:21.080291 9055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:21.080440 9055 raft_consensus.cc:686] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [term 1 LEADER]: Becoming Leader. State: Replica: e083ad897da442059331d3fdebede5fe, State: Running, Role: LEADER
I20260811 07:11:21.080498 9045 sys_catalog.cc:564] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:21.080570 9055 consensus_queue.cc:237] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } }
I20260811 07:11:21.080993 9055 sys_catalog.cc:455] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader e083ad897da442059331d3fdebede5fe. Latest consensus state: current_term: 1 leader_uuid: "e083ad897da442059331d3fdebede5fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } } }
I20260811 07:11:21.081090 9058 sys_catalog.cc:455] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e083ad897da442059331d3fdebede5fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e083ad897da442059331d3fdebede5fe" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42285 } } }
I20260811 07:11:21.081125 9055 sys_catalog.cc:458] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:21.081153 9058 sys_catalog.cc:458] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:21.081442 9066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:21.081976 9066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:21.084086 9066 catalog_manager.cc:1347] Generated new cluster ID: 40f1321c7803418593b8b9fadc2b248a
I20260811 07:11:21.084139 9066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:11:21 dist-test-slave-2d19 krb5kdc[8900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:21.089246 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:21.076937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48459 (local address 127.3.236.190:42285)
0811 07:11:21.077151 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:21.077162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:21.077197 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:11:21.077699 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:21.077706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:21.077757 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:21.077912 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:21.078577 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:21.079949 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:21.081219 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:21.081826 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:21.081935 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:21.085138 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:21.085143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:21.085143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:21.085156 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:21.087308 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:21.087876 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:21.087879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:21.087880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:21.087995 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:21.088248 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:21.088288 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:21.088288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:21.088622 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:21.088805 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:21.088922 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 07:11:21.089038 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
I20260811 07:11:21.090339 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:21.095306 9066 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:21.095825 9066 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:21.100049 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:21.091941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52953 (local address 127.3.236.190:42285)
0811 07:11:21.091982 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:21.091983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:21.092131 (+ 148us) server_negotiation.cc:407] Connection header received
0811 07:11:21.092178 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:21.092179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:21.092189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:21.092245 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:21.092924 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:21.093645 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:21.094634 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:21.094912 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:21.094996 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:21.097177 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:21.097178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:21.097178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:21.097186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:21.098803 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:21.099222 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:21.099223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:21.099224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:21.099285 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:21.099512 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:21.099513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:21.099513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:21.099640 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:21.099745 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:21.099890 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:11:21.099955 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:11:21.104108 9066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e083ad897da442059331d3fdebede5fe: Generated new TSK 0
I20260811 07:11:21.104310 9066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:21.106891 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 8923
W20260811 07:11:21.127627 8898 connection.cc:535] client connection to 127.3.236.190:42285 recv error: Network error: failed to read from TLS socket (remote: 127.3.236.190:42285): Connection reset by peer (error 104)
2026-08-11T07:11:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:21 dist-test-slave-2d19 krb5kdc[9103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:21 dist-test-slave-2d19 krb5kdc[9103](info): set up 2 sockets
Aug 11 07:11:21 dist-test-slave-2d19 krb5kdc[9103](info): commencing operation
krb5kdc: starting...
W20260811 07:11:22.429329 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:22 dist-test-slave-2d19 krb5kdc[9103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:22.556071 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46377
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:22.639314 9125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:22.639524 9125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:22.639546 9125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:22.640725 9125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:22.640787 9125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:22.640805 9125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:22.640816 9125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:22.640828 9125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:22.642334 9125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:22.642556 9125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:22.642735 9125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:22.647624 9136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:22.647639 9138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:22.647639 9141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:22.647877 9125 server_base.cc:1034] running on GCE node
I20260811 07:11:22.648188 9125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:22.648538 9125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:22.649730 9125 hybrid_clock.cc:648] HybridClock initialized: now 1786432282649706 us; error 35 us; skew 500 ppm
Aug 11 07:11:22 dist-test-slave-2d19 krb5kdc[9103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432282, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:22.652387 9125 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:22.652840 9125 webserver.cc:460] Webserver started at http://127.3.236.190:36077/ using document root <none> and password file <none>
I20260811 07:11:22.653069 9125 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:22.653112 9125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:22.653252 9125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:22.654304 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "ed5a634003df4226bbdb07cc9362f226"
format_stamp: "Formatted at 2026-08-11 07:11:22 on dist-test-slave-2d19"
server_key: "22a96c7175b095b5728649d7c4df8409"
server_key_iv: "463903033cad8bc31b06f9da52bda408"
server_key_version: "encryptionkey@0"
I20260811 07:11:22.654676 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "ed5a634003df4226bbdb07cc9362f226"
format_stamp: "Formatted at 2026-08-11 07:11:22 on dist-test-slave-2d19"
server_key: "22a96c7175b095b5728649d7c4df8409"
server_key_iv: "463903033cad8bc31b06f9da52bda408"
server_key_version: "encryptionkey@0"
I20260811 07:11:22.655992 9125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 07:11:22.656814 9150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:22.657040 9125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 07:11:22.657119 9125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "ed5a634003df4226bbdb07cc9362f226"
format_stamp: "Formatted at 2026-08-11 07:11:22 on dist-test-slave-2d19"
server_key: "22a96c7175b095b5728649d7c4df8409"
server_key_iv: "463903033cad8bc31b06f9da52bda408"
server_key_version: "encryptionkey@0"
I20260811 07:11:22.657213 9125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:22.674939 9125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:22.676925 9125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:22.677100 9125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:22.681355 9125 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:46377
I20260811 07:11:22.681404 9237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:46377 every 8 connection(s)
I20260811 07:11:22.681803 9125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:22.682693 9240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:22.685120 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226: Bootstrap starting.
I20260811 07:11:22.685729 9240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:22.686038 9240 log.cc:824] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:22.686757 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226: No bootstrap required, opened a new log
I20260811 07:11:22.688473 9240 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } }
I20260811 07:11:22.688635 9240 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:22.688659 9240 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed5a634003df4226bbdb07cc9362f226, State: Initialized, Role: FOLLOWER
I20260811 07:11:22.688827 9240 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } }
I20260811 07:11:22.688890 9240 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:22.688933 9240 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:22.688985 9240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:22.689870 9240 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } }
I20260811 07:11:22.689993 9240 leader_election.cc:302] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed5a634003df4226bbdb07cc9362f226; no voters:
I20260811 07:11:22.690183 9240 leader_election.cc:288] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:22.690218 9244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:22.690395 9244 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [term 1 LEADER]: Becoming Leader. State: Replica: ed5a634003df4226bbdb07cc9362f226, State: Running, Role: LEADER
I20260811 07:11:22.690456 9240 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:22.690536 9244 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } }
I20260811 07:11:22.690950 9245 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed5a634003df4226bbdb07cc9362f226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } } }
I20260811 07:11:22.690953 9247 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed5a634003df4226bbdb07cc9362f226. Latest consensus state: current_term: 1 leader_uuid: "ed5a634003df4226bbdb07cc9362f226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5a634003df4226bbdb07cc9362f226" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46377 } } }
I20260811 07:11:22.691164 9247 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:22.691087 9245 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:22.691618 9254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:22.692059 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9125
I20260811 07:11:22.692199 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:22.692346 9254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:22.692415 4018 external_mini_cluster.cc:1428] Setting key 0883465b5f9abf9f58ac63fdeef5ae23
I20260811 07:11:22.694036 9254 catalog_manager.cc:1347] Generated new cluster ID: f30cad4f7a7946c4b72df3bb43591277
I20260811 07:11:22.694093 9254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:22.698293 9254 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:22.698827 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:22.693074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36399 (local address 127.3.236.190:46377)
0811 07:11:22.693236 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:22.693247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:22.693279 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:11:22.693387 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:22.693392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:22.693487 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:22.693672 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:22.693892 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:22.695316 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:22.696449 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:22.697062 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:22.697190 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:22.698006 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:22.698011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:22.698011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:22.698262 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:22.698306 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:22.698321 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:22.698451 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:22.698536 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 07:11:22.698676 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260811 07:11:22.699016 9254 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:22.699699 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:22.705687 9254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed5a634003df4226bbdb07cc9362f226: Generated new TSK 0
I20260811 07:11:22.705858 9254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:22.707800 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:22.701149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34047 (local address 127.3.236.190:46377)
0811 07:11:22.701187 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:22.701189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:22.704457 (+ 3268us) server_negotiation.cc:407] Connection header received
0811 07:11:22.704532 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:22.704534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:22.704543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:22.704592 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:22.704775 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:22.705585 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:22.706441 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:22.706773 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:22.706847 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:22.707523 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:22.707524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:22.707525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:22.707529 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:22.707545 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:22.707549 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:22.707614 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:22.707710 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 07:11:22.707721 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:11:22.709892 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9125
W20260811 07:11:22.728292 9098 connection.cc:535] client connection to 127.3.236.190:46377 recv error: Network error: failed to read from TLS socket (remote: 127.3.236.190:46377): Connection reset by peer (error 104)
2026-08-11T07:11:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1600 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:22 dist-test-slave-2d19 krb5kdc[9289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:22 dist-test-slave-2d19 krb5kdc[9289](info): set up 2 sockets
Aug 11 07:11:22 dist-test-slave-2d19 krb5kdc[9289](info): commencing operation
krb5kdc: starting...
W20260811 07:11:24.033715 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:24 dist-test-slave-2d19 krb5kdc[9289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:24.163166 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:32941
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:32941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:24.246744 9309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:24.246948 9309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:24.246970 9309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:24.248123 9309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:24.248170 9309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:24.248184 9309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:24.248198 9309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:24.248267 9309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:24.249737 9309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:32941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:32941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:24.249993 9309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:24.250226 9309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:24.255264 9329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:24.255273 9326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:24.255364 9324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:24.255522 9309 server_base.cc:1034] running on GCE node
I20260811 07:11:24.255856 9309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:24.256155 9309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:24.257300 9309 hybrid_clock.cc:648] HybridClock initialized: now 1786432284257292 us; error 33 us; skew 500 ppm
Aug 11 07:11:24 dist-test-slave-2d19 krb5kdc[9289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432284, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:24.259773 9309 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:24.260332 9309 webserver.cc:460] Webserver started at http://127.3.236.190:39335/ using document root <none> and password file <none>
I20260811 07:11:24.260560 9309 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:24.260627 9309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:24.260787 9309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:24.261842 9309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "9e9917a596d74f18972e424fd2ce364e"
format_stamp: "Formatted at 2026-08-11 07:11:24 on dist-test-slave-2d19"
server_key: "b7fdf85f09a3f3734c86dd98ccba6ca7"
server_key_iv: "56c2821ccf4cf2c108c8a3ada578caa6"
server_key_version: "encryptionkey@0"
I20260811 07:11:24.262246 9309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "9e9917a596d74f18972e424fd2ce364e"
format_stamp: "Formatted at 2026-08-11 07:11:24 on dist-test-slave-2d19"
server_key: "b7fdf85f09a3f3734c86dd98ccba6ca7"
server_key_iv: "56c2821ccf4cf2c108c8a3ada578caa6"
server_key_version: "encryptionkey@0"
I20260811 07:11:24.263664 9309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:24.264451 9340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:24.264627 9309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 07:11:24.264739 9309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "9e9917a596d74f18972e424fd2ce364e"
format_stamp: "Formatted at 2026-08-11 07:11:24 on dist-test-slave-2d19"
server_key: "b7fdf85f09a3f3734c86dd98ccba6ca7"
server_key_iv: "56c2821ccf4cf2c108c8a3ada578caa6"
server_key_version: "encryptionkey@0"
I20260811 07:11:24.264812 9309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:24.268070 9309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:24.269925 9309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:24.270092 9309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:24.274222 9309 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:32941
I20260811 07:11:24.274253 9429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:32941 every 8 connection(s)
I20260811 07:11:24.274576 9309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:24.275552 9432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:24.277859 9432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e: Bootstrap starting.
I20260811 07:11:24.278142 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9309
I20260811 07:11:24.278282 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:24.278512 4018 external_mini_cluster.cc:1428] Setting key 9dd7d2752389d95966acf7b2e690468d
I20260811 07:11:24.278602 9432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:24.278970 9432 log.cc:824] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:24.279835 9432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e: No bootstrap required, opened a new log
I20260811 07:11:24.281394 9437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:24.279070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60351 (local address 127.3.236.190:32941)
0811 07:11:24.279289 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:24.279298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:24.279344 (+ 46us) server_negotiation.cc:407] Connection header received
0811 07:11:24.279989 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:24.280000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:24.280035 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:24.280182 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:24.280923 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:24.280927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:24.280928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:24.280933 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:24.280977 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:24.280992 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:24.281098 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:24.281130 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 07:11:24.281171 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260811 07:11:24.281812 9432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } }
I20260811 07:11:24.281960 9432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:24.281982 9432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9917a596d74f18972e424fd2ce364e, State: Initialized, Role: FOLLOWER
I20260811 07:11:24.282131 9432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } }
I20260811 07:11:24.282228 9432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:24.282262 9432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:24.282310 9432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:24.282802 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:24.283007 9432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } }
I20260811 07:11:24.283131 9432 leader_election.cc:302] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e9917a596d74f18972e424fd2ce364e; no voters:
I20260811 07:11:24.283330 9432 leader_election.cc:288] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:24.283389 9440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:24.283586 9440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [term 1 LEADER]: Becoming Leader. State: Replica: 9e9917a596d74f18972e424fd2ce364e, State: Running, Role: LEADER
I20260811 07:11:24.283597 9432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:24.283726 9440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } }
I20260811 07:11:24.284101 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e9917a596d74f18972e424fd2ce364e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } } }
I20260811 07:11:24.284212 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:24.284089 9440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e9917a596d74f18972e424fd2ce364e. Latest consensus state: current_term: 1 leader_uuid: "9e9917a596d74f18972e424fd2ce364e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e9917a596d74f18972e424fd2ce364e" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 32941 } } }
I20260811 07:11:24.284449 9440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:24.284512 9457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:24.285044 9457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:24.286698 9457 catalog_manager.cc:1347] Generated new cluster ID: 00c2c014484a4f4496e83ef1fbe5677b
I20260811 07:11:24.286760 9457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:24.287046 9437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:24.285741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50211 (local address 127.3.236.190:32941)
0811 07:11:24.285774 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:24.285776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:24.285923 (+ 147us) server_negotiation.cc:407] Connection header received
0811 07:11:24.286023 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:24.286023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:24.286029 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:24.286058 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:24.286804 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:24.286806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:24.286806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:24.286811 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:24.286824 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:24.286829 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:24.286859 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:24.286970 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 07:11:24.286984 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 07:11:24.301735 9457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:24.301913 9457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:24.308688 9457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e9917a596d74f18972e424fd2ce364e: Generated new TSK 0
I20260811 07:11:24.308907 9457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:24.319492 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9309
2026-08-11T07:11:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1608 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:24 dist-test-slave-2d19 krb5kdc[9485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:24 dist-test-slave-2d19 krb5kdc[9485](info): set up 2 sockets
Aug 11 07:11:24 dist-test-slave-2d19 krb5kdc[9485](info): commencing operation
krb5kdc: starting...
W20260811 07:11:25.610337 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:25.740479 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35891
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:42073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:35891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:25.824738 9513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:25.824954 9513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:25.824978 9513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:25.826076 9513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:25.826112 9513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:25.826128 9513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:25.826139 9513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:25.826152 9513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:25.827565 9513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:42073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:35891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:35891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:25.827750 9513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:25.827937 9513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:25.833002 9525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:25.833002 9522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:25.833297 9523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:25.833401 9513 server_base.cc:1034] running on GCE node
I20260811 07:11:25.833623 9513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:25.833870 9513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:25.835040 9513 hybrid_clock.cc:648] HybridClock initialized: now 1786432285835028 us; error 26 us; skew 500 ppm
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:25.841920 9513 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:25.842453 9513 webserver.cc:460] Webserver started at http://127.3.236.190:40749/ using document root <none> and password file <none>
I20260811 07:11:25.842720 9513 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:25.842792 9513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:25.842943 9513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:25.844050 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "91494df5c3094437b4f1dbd9e3619411"
format_stamp: "Formatted at 2026-08-11 07:11:25 on dist-test-slave-2d19"
server_key: "aadd5384304d134fd225fe8458cbe9db"
server_key_iv: "be4779763ba171f0632a7004b78e1007"
server_key_version: "encryptionkey@0"
I20260811 07:11:25.844465 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "91494df5c3094437b4f1dbd9e3619411"
format_stamp: "Formatted at 2026-08-11 07:11:25 on dist-test-slave-2d19"
server_key: "aadd5384304d134fd225fe8458cbe9db"
server_key_iv: "be4779763ba171f0632a7004b78e1007"
server_key_version: "encryptionkey@0"
I20260811 07:11:25.845870 9513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:25.846683 9538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:25.846868 9513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:25.847009 9513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "91494df5c3094437b4f1dbd9e3619411"
format_stamp: "Formatted at 2026-08-11 07:11:25 on dist-test-slave-2d19"
server_key: "aadd5384304d134fd225fe8458cbe9db"
server_key_iv: "be4779763ba171f0632a7004b78e1007"
server_key_version: "encryptionkey@0"
I20260811 07:11:25.847092 9513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:25.862779 9513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:25.865214 9513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:25.865398 9513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:25.870318 9513 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:35891
I20260811 07:11:25.870355 9621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:35891 every 8 connection(s)
I20260811 07:11:25.870730 9513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:25.871680 9623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:25.874256 9623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411: Bootstrap starting.
I20260811 07:11:25.874890 9623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:25.875216 9623 log.cc:824] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:25.876022 9623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411: No bootstrap required, opened a new log
I20260811 07:11:25.876493 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9513
I20260811 07:11:25.876629 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:25.876829 4018 external_mini_cluster.cc:1428] Setting key 80f779ae1a673965f80fd4ae72e1c3f1
I20260811 07:11:25.878429 9623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } }
I20260811 07:11:25.878612 9623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:25.878664 9623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91494df5c3094437b4f1dbd9e3619411, State: Initialized, Role: FOLLOWER
I20260811 07:11:25.878815 9623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } }
I20260811 07:11:25.878919 9623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:25.878958 9623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:25.879001 9623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:25.880125 9623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } }
I20260811 07:11:25.880221 9623 leader_election.cc:302] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91494df5c3094437b4f1dbd9e3619411; no voters:
I20260811 07:11:25.880431 9623 leader_election.cc:288] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:25.880481 9630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:25.880636 9630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [term 1 LEADER]: Becoming Leader. State: Replica: 91494df5c3094437b4f1dbd9e3619411, State: Running, Role: LEADER
I20260811 07:11:25.880716 9623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:25.880723 9630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } }
I20260811 07:11:25.881115 9630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91494df5c3094437b4f1dbd9e3619411. Latest consensus state: current_term: 1 leader_uuid: "91494df5c3094437b4f1dbd9e3619411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } } }
I20260811 07:11:25.881203 9630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:25.881145 9632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91494df5c3094437b4f1dbd9e3619411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91494df5c3094437b4f1dbd9e3619411" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 35891 } } }
I20260811 07:11:25.881409 9640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:25.882040 9640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:25.881390 9632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411 [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:25.883737 9640 catalog_manager.cc:1347] Generated new cluster ID: 4661bb292a5f45a6bd26cfa31d91bc63
I20260811 07:11:25.883792 9640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:25.888993 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:25.877393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36735 (local address 127.3.236.190:35891)
0811 07:11:25.877609 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:25.877618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:25.877646 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:11:25.878410 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:25.878419 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:25.878475 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:25.878618 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:25.879627 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:25.880805 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:25.882034 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:25.882660 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:25.882770 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:25.885518 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:25.885522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:25.885523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:25.885536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:25.887295 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:25.887706 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:25.887710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:25.887710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:25.887832 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:25.888062 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:25.888063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:25.888063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:25.888366 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:25.888557 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:25.888632 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 07:11:25.888780 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":112,"threads_started":1}
I20260811 07:11:25.889979 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:25.894982 9640 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:25.895707 9640 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:25.899194 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:25.891390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56373 (local address 127.3.236.190:35891)
0811 07:11:25.891431 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:25.891433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:25.892466 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 07:11:25.892544 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:25.892545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:25.892556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:25.892611 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:25.893167 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:25.893927 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:25.894723 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:25.895056 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:25.895122 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:25.896498 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:25.896499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:25.896500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:25.896508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:25.897974 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:25.898343 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:25.898345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:25.898345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:25.898415 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:25.898630 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:25.898631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:25.898631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:25.898744 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:25.898846 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:25.898988 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 07:11:25.899080 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":215,"server-negotiator.queue_time_us":21}
I20260811 07:11:25.904284 9640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91494df5c3094437b4f1dbd9e3619411: Generated new TSK 0
I20260811 07:11:25.904471 9640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:25.906762 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9513
2026-08-11T07:11:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9673](info): set up 2 sockets
Aug 11 07:11:25 dist-test-slave-2d19 krb5kdc[9673](info): commencing operation
krb5kdc: starting...
W20260811 07:11:27.262960 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:27 dist-test-slave-2d19 krb5kdc[9673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:27.391927 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38693
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:38693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:27.478947 9694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:27.479212 9694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:27.479255 9694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:27.480454 9694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:27.480527 9694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:27.480551 9694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:27.480572 9694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:27.480600 9694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:27.482126 9694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:38693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:38693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:27.482409 9694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:27.482648 9694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:27.488044 9709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:27.488039 9706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:27.488065 9705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:27.488528 9694 server_base.cc:1034] running on GCE node
I20260811 07:11:27.488813 9694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:27.489140 9694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:27.490314 9694 hybrid_clock.cc:648] HybridClock initialized: now 1786432287490301 us; error 51 us; skew 500 ppm
Aug 11 07:11:27 dist-test-slave-2d19 krb5kdc[9673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:27.492992 9694 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:27.493451 9694 webserver.cc:460] Webserver started at http://127.3.236.190:34243/ using document root <none> and password file <none>
I20260811 07:11:27.493683 9694 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:27.493753 9694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:27.493906 9694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:27.494982 9694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "fe4febc141ca46548d45d82e29ed320c"
format_stamp: "Formatted at 2026-08-11 07:11:27 on dist-test-slave-2d19"
server_key: "e2e6d331c4793ec63ec22ef2fb0e84de"
server_key_iv: "2b009efdb55cb92cb86e24fb6cc94120"
server_key_version: "encryptionkey@0"
I20260811 07:11:27.495368 9694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "fe4febc141ca46548d45d82e29ed320c"
format_stamp: "Formatted at 2026-08-11 07:11:27 on dist-test-slave-2d19"
server_key: "e2e6d331c4793ec63ec22ef2fb0e84de"
server_key_iv: "2b009efdb55cb92cb86e24fb6cc94120"
server_key_version: "encryptionkey@0"
I20260811 07:11:27.496837 9694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:27.497608 9723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:27.497800 9694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:27.497910 9694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "fe4febc141ca46548d45d82e29ed320c"
format_stamp: "Formatted at 2026-08-11 07:11:27 on dist-test-slave-2d19"
server_key: "e2e6d331c4793ec63ec22ef2fb0e84de"
server_key_iv: "2b009efdb55cb92cb86e24fb6cc94120"
server_key_version: "encryptionkey@0"
I20260811 07:11:27.497983 9694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:27.505890 9694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:27.507679 9694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:27.507836 9694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:27.512141 9694 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:38693
I20260811 07:11:27.512207 9810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:38693 every 8 connection(s)
I20260811 07:11:27.512588 9694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:27.513589 9811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:27.515807 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: Bootstrap starting.
I20260811 07:11:27.516425 9811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:27.516716 9811 log.cc:824] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:27.517376 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: No bootstrap required, opened a new log
I20260811 07:11:27.518111 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9694
I20260811 07:11:27.518239 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:27.518411 4018 external_mini_cluster.cc:1428] Setting key c8ccf91bee5314ec14e804d8d124aef4
I20260811 07:11:27.519531 9811 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } }
I20260811 07:11:27.519706 9811 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:27.519747 9811 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe4febc141ca46548d45d82e29ed320c, State: Initialized, Role: FOLLOWER
I20260811 07:11:27.519899 9811 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } }
I20260811 07:11:27.520005 9811 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:27.520051 9811 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:27.520098 9811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:27.520784 9811 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } }
I20260811 07:11:27.520908 9811 leader_election.cc:302] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe4febc141ca46548d45d82e29ed320c; no voters:
I20260811 07:11:27.521082 9811 leader_election.cc:288] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:27.521108 9826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:27.521226 9826 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [term 1 LEADER]: Becoming Leader. State: Replica: fe4febc141ca46548d45d82e29ed320c, State: Running, Role: LEADER
I20260811 07:11:27.521355 9811 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:27.521368 9826 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } }
I20260811 07:11:27.521812 9830 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe4febc141ca46548d45d82e29ed320c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } } }
I20260811 07:11:27.521826 9831 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4febc141ca46548d45d82e29ed320c. Latest consensus state: current_term: 1 leader_uuid: "fe4febc141ca46548d45d82e29ed320c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4febc141ca46548d45d82e29ed320c" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 38693 } } }
I20260811 07:11:27.521968 9830 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:27.522068 9831 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c [sys.catalog]: This master's current role is: LEADER
W20260811 07:11:27.523423 9856 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:27.523526 9856 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:27.523598 9843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:27.523758 9843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:27.524878 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:27.518929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36671 (local address 127.3.236.190:38693)
0811 07:11:27.519151 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:27.519161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:27.519194 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:11:27.519485 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:27.519493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:27.519555 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:27.519713 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:27.519949 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:27.521424 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:27.522531 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:27.523273 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:27.523397 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:27.524122 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:27.524126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:27.524127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:27.524389 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:27.524424 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:27.524436 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:27.524515 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:27.524592 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 07:11:27.524669 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260811 07:11:27.525491 9843 catalog_manager.cc:1347] Generated new cluster ID: f9bcd557172c4a78af4360472f7fae59
I20260811 07:11:27.525542 9843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:27.525848 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:27.530804 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:27.527257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45029 (local address 127.3.236.190:38693)
0811 07:11:27.527317 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:27.527319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:27.527531 (+ 212us) server_negotiation.cc:407] Connection header received
0811 07:11:27.527558 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:27.527559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:27.527570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:27.527639 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:27.527775 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:27.528502 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:27.529497 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:27.529813 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:27.529894 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:27.530566 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:27.530567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:27.530568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:27.530572 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:27.530587 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:27.530592 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:27.530653 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:27.530710 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 07:11:27.530733 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:11:27.538183 9843 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:27.538722 9843 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:27.541356 9843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe4febc141ca46548d45d82e29ed320c: Generated new TSK 0
I20260811 07:11:27.541533 9843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:27.549858 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9694
2026-08-11T07:11:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:27 dist-test-slave-2d19 krb5kdc[9876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:27 dist-test-slave-2d19 krb5kdc[9876](info): set up 2 sockets
Aug 11 07:11:27 dist-test-slave-2d19 krb5kdc[9876](info): commencing operation
krb5kdc: starting...
W20260811 07:11:28.948045 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:29 dist-test-slave-2d19 krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:29.080041 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42203
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:42203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:29.165261 9899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:29.165477 9899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:29.165501 9899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:29.166591 9899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:29.166628 9899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:29.166642 9899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:29.166656 9899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:29.166667 9899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:29.168210 9899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:42203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:29.168445 9899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:29.168624 9899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:29.173666 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:29.173666 9915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:29.173717 9899 server_base.cc:1034] running on GCE node
W20260811 07:11:29.173697 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:29.174348 9899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:29.174634 9899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:29.175798 9899 hybrid_clock.cc:648] HybridClock initialized: now 1786432289175784 us; error 34 us; skew 500 ppm
Aug 11 07:11:29 dist-test-slave-2d19 krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432289, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:29.178651 9899 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:29.179051 9899 webserver.cc:460] Webserver started at http://127.3.236.190:37979/ using document root <none> and password file <none>
I20260811 07:11:29.179304 9899 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:29.179351 9899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:29.179539 9899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:29.180804 9899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "ef72bd5340684e558d115db3fe635abb"
format_stamp: "Formatted at 2026-08-11 07:11:29 on dist-test-slave-2d19"
server_key: "e10b634cd34b92ae1a2dd48c2795eec4"
server_key_iv: "d7c3d394c15b77ba43c4e2f2550ed7ed"
server_key_version: "encryptionkey@0"
I20260811 07:11:29.181178 9899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "ef72bd5340684e558d115db3fe635abb"
format_stamp: "Formatted at 2026-08-11 07:11:29 on dist-test-slave-2d19"
server_key: "e10b634cd34b92ae1a2dd48c2795eec4"
server_key_iv: "d7c3d394c15b77ba43c4e2f2550ed7ed"
server_key_version: "encryptionkey@0"
I20260811 07:11:29.182578 9899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 07:11:29.183382 9926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:29.183622 9899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:29.183701 9899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "ef72bd5340684e558d115db3fe635abb"
format_stamp: "Formatted at 2026-08-11 07:11:29 on dist-test-slave-2d19"
server_key: "e10b634cd34b92ae1a2dd48c2795eec4"
server_key_iv: "d7c3d394c15b77ba43c4e2f2550ed7ed"
server_key_version: "encryptionkey@0"
I20260811 07:11:29.183792 9899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:29.187214 9899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:29.189443 9899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:29.189608 9899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:29.193722 9899 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:42203
I20260811 07:11:29.193753 10015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:42203 every 8 connection(s)
I20260811 07:11:29.194087 9899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:29.195108 10016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:29.195672 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 9899
I20260811 07:11:29.195801 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:29.195983 4018 external_mini_cluster.cc:1428] Setting key cb214966f961b8843007fea60dbfc4ee
I20260811 07:11:29.198278 10016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb: Bootstrap starting.
I20260811 07:11:29.198596 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:29.196662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34647 (local address 127.3.236.190:42203)
0811 07:11:29.196845 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:29.196854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:29.196879 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:11:29.197326 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:29.197333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:29.197360 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:29.197471 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:29.198117 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:29.198120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:29.198120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:29.198124 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:29.198160 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:29.198170 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:29.198228 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:29.198324 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 07:11:29.198385 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260811 07:11:29.199038 10016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:29.199388 10016 log.cc:824] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:29.199580 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:29.200114 10016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb: No bootstrap required, opened a new log
I20260811 07:11:29.202167 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:29.201115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39663 (local address 127.3.236.190:42203)
0811 07:11:29.201166 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:29.201168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:29.201231 (+ 63us) server_negotiation.cc:407] Connection header received
0811 07:11:29.201309 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:29.201310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:29.201314 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:29.201366 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:29.201950 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:29.201951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:29.201952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:11:29.201957 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:29.201971 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:11:29.201975 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:29.202031 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:29.202086 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 07:11:29.202101 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:11:29.202390 10016 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } }
I20260811 07:11:29.202575 10016 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:11:29.202631 10016 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef72bd5340684e558d115db3fe635abb, State: Initialized, Role: FOLLOWER
I20260811 07:11:29.202761 10016 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } }
I20260811 07:11:29.202849 10016 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:11:29.202872 10016 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:11:29.202901 10016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:29.203389 10016 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } }
I20260811 07:11:29.203485 10016 leader_election.cc:302] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef72bd5340684e558d115db3fe635abb; no voters:
I20260811 07:11:29.203603 10016 leader_election.cc:288] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:11:29.203677 10034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:29.203825 10034 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [term 1 LEADER]: Becoming Leader. State: Replica: ef72bd5340684e558d115db3fe635abb, State: Running, Role: LEADER
I20260811 07:11:29.203836 10016 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:29.204008 10034 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } }
I20260811 07:11:29.204355 10035 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef72bd5340684e558d115db3fe635abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } } }
I20260811 07:11:29.204363 10037 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef72bd5340684e558d115db3fe635abb. Latest consensus state: current_term: 1 leader_uuid: "ef72bd5340684e558d115db3fe635abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef72bd5340684e558d115db3fe635abb" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 42203 } } }
I20260811 07:11:29.204519 10035 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:29.204530 10037 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:29.205165 10045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:29.205655 10045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:29.207207 10045 catalog_manager.cc:1347] Generated new cluster ID: 096886656131441786583b1e1a2d5533
I20260811 07:11:29.207257 10045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:29.240432 10045 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:29.240661 10045 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:29.245564 10045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef72bd5340684e558d115db3fe635abb: Generated new TSK 0
I20260811 07:11:29.245761 10045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:11:29.248260 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 9899
2026-08-11T07:11:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1690 ms)
[----------] 18 tests from AuthTokenIssuingTest (28877 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:29 dist-test-slave-2d19 krb5kdc[10064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:29 dist-test-slave-2d19 krb5kdc[10064](info): set up 2 sockets
Aug 11 07:11:29 dist-test-slave-2d19 krb5kdc[10064](info): commencing operation
krb5kdc: starting...
W20260811 07:11:30.600135 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:30 dist-test-slave-2d19 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:30.726913 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:33075
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:30.811830 10086 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:30.812088 10086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:30.812106 10086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:30.812129 10086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:30.813330 10086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:30.813373 10086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:30.813387 10086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:30.813400 10086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:30.813454 10086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:30.814956 10086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:33075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:30.815229 10086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:30.815464 10086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:30.820485 10106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:30.820588 10100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:30.820555 10086 server_base.cc:1034] running on GCE node
W20260811 07:11:30.820497 10099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:30.820912 10086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:30.821164 10086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:30.822305 10086 hybrid_clock.cc:648] HybridClock initialized: now 1786432290822287 us; error 38 us; skew 500 ppm
Aug 11 07:11:30 dist-test-slave-2d19 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:30.824841 10086 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:30.825323 10086 webserver.cc:460] Webserver started at http://127.3.236.190:40181/ using document root <none> and password file <none>
I20260811 07:11:30.825577 10086 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:30.825623 10086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:30.825788 10086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:30.826874 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "f978d7e4308e4b4eb814a86e2d17440d"
format_stamp: "Formatted at 2026-08-11 07:11:30 on dist-test-slave-2d19"
server_key: "a86c53d28da950891a0cfd073a1bfd76"
server_key_iv: "fc00e4b9ef0c7c56455375a72f2d53f6"
server_key_version: "encryptionkey@0"
I20260811 07:11:30.827243 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "f978d7e4308e4b4eb814a86e2d17440d"
format_stamp: "Formatted at 2026-08-11 07:11:30 on dist-test-slave-2d19"
server_key: "a86c53d28da950891a0cfd073a1bfd76"
server_key_iv: "fc00e4b9ef0c7c56455375a72f2d53f6"
server_key_version: "encryptionkey@0"
I20260811 07:11:30.828716 10086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:30.829463 10118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:30.829644 10086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:30.829737 10086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "f978d7e4308e4b4eb814a86e2d17440d"
format_stamp: "Formatted at 2026-08-11 07:11:30 on dist-test-slave-2d19"
server_key: "a86c53d28da950891a0cfd073a1bfd76"
server_key_iv: "fc00e4b9ef0c7c56455375a72f2d53f6"
server_key_version: "encryptionkey@0"
I20260811 07:11:30.829793 10086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:30.878328 10086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:30.880416 10086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:30.880535 10086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:30.884600 10086 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:33075
I20260811 07:11:30.884696 10211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:33075 every 8 connection(s)
I20260811 07:11:30.885051 10086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:30.886121 10214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:30.887698 10214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:30.892832 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10086
I20260811 07:11:30.892962 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:30.893179 4018 external_mini_cluster.cc:1428] Setting key 824679f8a7837aa33026d72d1031d75c
Aug 11 07:11:30 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:30.897940 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.888585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.3.236.190:33075)
0811 07:11:30.888711 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:30.888715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:30.888748 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:11:30.889259 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:30.889261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:30.889278 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:30.889358 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:30.889953 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:30.890802 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:30.891683 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:30.892011 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:30.892103 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:30.894547 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:30.894556 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:30.894556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:30.894564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:30.896331 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:30.896808 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:30.896811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:30.896811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:30.896891 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:30.897148 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:30.897149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:30.897149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:30.897358 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:30.897423 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:30.897671 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 07:11:30.897767 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260811 07:11:30.898082 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.888425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:33075 (local address 127.0.0.1:40744)
0811 07:11:30.888587 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:30.888625 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:30.889119 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:30.889392 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:30.889395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:30.889655 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:30.889827 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:30.889831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:30.890896 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:30.890897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:30.891590 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:30.891592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:30.892120 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:30.892121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:30.892175 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:30.892663 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:30.892672 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:30.894449 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:30.896419 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:30.896435 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:30.896438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:30.896719 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:30.896957 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:30.896958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:30.896958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:30.897078 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:30.897438 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:30.897446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:30.897527 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 07:11:30.897633 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 07:11:30.897899 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":74,"thread_start_us":70,"threads_started":1}
I20260811 07:11:30.898995 10214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:30.899403 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.899253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52792)
0811 07:11:30.899310 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:30.899368 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:11:30.899580 10119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
I20260811 07:11:30.900076 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.899956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52800)
0811 07:11:30.900009 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:30.900044 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 07:11:30.900228 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36657: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
I20260811 07:11:30.923229 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:36657
--webserver_interface=127.3.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:11:30.949505 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } attempt: 1
I20260811 07:11:30.950111 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.949938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52810)
0811 07:11:30.950014 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:30.950071 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:11:30.950604 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:30.950493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52822)
0811 07:11:30.950544 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:30.950579 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 07:11:30.950770 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36657: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
W20260811 07:11:31.014158 10224 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:31.014370 10224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:31.014387 10224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:31.014407 10224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:31.015565 10224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:31.015607 10224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:31.015622 10224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:31.015635 10224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:31.015693 10224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:31.017385 10224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:36657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:31.017652 10224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:31.017954 10224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:11:31.018970 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } attempt: 2
I20260811 07:11:31.019457 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.019313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52830)
0811 07:11:31.019384 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.019426 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:11:31.019871 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.019770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52842)
0811 07:11:31.019819 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.019845 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:11:31.019968 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:36657: Network error: Client connection negotiation failed: client connection to 127.3.236.189:36657: connect: Connection refused (error 111)
W20260811 07:11:31.023026 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:31.023082 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:31.023044 10235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:31.023372 10224 server_base.cc:1034] running on GCE node
I20260811 07:11:31.023623 10224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:31.023911 10224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:31.025076 10224 hybrid_clock.cc:648] HybridClock initialized: now 1786432291025043 us; error 49 us; skew 500 ppm
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:31.027624 10224 init.cc:377] Logged in from keytab as kudu/127.3.236.189@KRBTEST.COM (short username kudu)
I20260811 07:11:31.028096 10224 webserver.cc:460] Webserver started at http://127.3.236.189:39857/ using document root <none> and password file <none>
I20260811 07:11:31.028391 10224 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:31.028463 10224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:31.028609 10224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:31.029706 10224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data/instance:
uuid: "273e38d86e4b4dd297662b5f93721721"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "697a3e5d387729c0848c0ea88ebaefef"
server_key_iv: "0045e4996ae24f29e894342d2a32324b"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.030079 10224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal/instance:
uuid: "273e38d86e4b4dd297662b5f93721721"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "697a3e5d387729c0848c0ea88ebaefef"
server_key_iv: "0045e4996ae24f29e894342d2a32324b"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.031486 10224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:31.032430 10256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:31.032671 10224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:31.032799 10224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "273e38d86e4b4dd297662b5f93721721"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "697a3e5d387729c0848c0ea88ebaefef"
server_key_iv: "0045e4996ae24f29e894342d2a32324b"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.032888 10224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:31.052803 10224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:31.054787 10224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:31.054966 10224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:31.061074 10224 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.189:36657
I20260811 07:11:31.061134 10340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.189:36657 every 8 connection(s)
I20260811 07:11:31.061514 10224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
I20260811 07:11:31.062628 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:31.064234 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.069435 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10224
I20260811 07:11:31.069561 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-1/wal/instance
I20260811 07:11:31.069756 4018 external_mini_cluster.cc:1428] Setting key 43501477125d03eaaea62482a490c5c5
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:31.073649 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.065038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.3.236.190:33075)
0811 07:11:31.065125 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.065128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.065250 (+ 122us) server_negotiation.cc:407] Connection header received
0811 07:11:31.065712 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.065713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.065724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.065777 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.066356 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.066837 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.067793 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.067991 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.068064 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.070494 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.070496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.070496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.070505 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.072177 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.072677 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.072678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.072678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.072731 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.073022 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.073022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.073022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.073129 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.073181 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.073518 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.073584 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:11:31.074044 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.064936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:33075 (local address 127.0.0.1:40748)
0811 07:11:31.065118 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.065163 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.065628 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.065831 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.065834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.066086 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.066284 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.066287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.066916 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.066917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.067715 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.067717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.068114 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.068115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.068241 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.068850 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.068859 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.070389 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.072273 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.072281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.072284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.072601 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.072800 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.072801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.072801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.072941 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.073197 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.073204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.073371 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.073498 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.073680 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260811 07:11:31.074712 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.236.188@KRBTEST.COM; defaulting to no policy
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.188@KRBTEST.COM" created.
I20260811 07:11:31.083201 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.074986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52844)
0811 07:11:31.075048 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.075059 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.075096 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.075375 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.075377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.075535 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.075601 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.075602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.077059 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.077060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.077682 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.077685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.078252 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.078253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.078326 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.078753 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.078758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.080172 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.081920 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.081921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.081921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.082175 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.082375 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.082376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.082376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.082432 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.082698 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.082698 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.082763 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.082859 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.082974 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:11:31.083207 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.075076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.3.236.189:36657)
0811 07:11:31.075224 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.075228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.075244 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:11:31.075263 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.075265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.075276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.075344 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.075708 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.076948 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.077801 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.078151 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.078252 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.080316 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.080324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.080324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.080331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.081844 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.082254 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.082258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.082258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.082312 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.082488 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.082489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.082490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.082640 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.082704 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.082914 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.083057 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260811 07:11:31.083877 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.084218 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.084100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51886)
0811 07:11:31.084148 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.084196 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 07:11:31.084378 10257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
I20260811 07:11:31.084837 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.084740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51896)
0811 07:11:31.084792 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.084816 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 07:11:31.084944 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
I20260811 07:11:31.099368 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:38909
--webserver_interface=127.3.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:11:31.111111 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } attempt: 3
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:11:31.118757 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.111544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.3.236.189:36657)
0811 07:11:31.111614 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.111617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.111628 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:11:31.111692 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.111693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.111705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.111777 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.112128 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.112687 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.113425 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.113825 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.113927 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.116102 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.116103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.116104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.116110 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.117609 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.118038 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.118038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.118039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.118090 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.118301 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.118301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.118302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.118399 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.118448 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.118621 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.118691 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:11:31.118824 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.111460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52850)
0811 07:11:31.111539 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.111549 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.111622 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.111787 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.111788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.111961 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.112036 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.112039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.112775 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.112776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.113302 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.113323 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.113925 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.113926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.113990 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.114505 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.114510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.116003 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.117698 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.117699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.117700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.117941 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.118160 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.118160 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.118161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.118235 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.118463 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.118464 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.118520 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.118597 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.118710 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:11:31.119560 10214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.119953 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.119825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51910)
0811 07:11:31.119882 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.119922 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 07:11:31.120333 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.120219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51912)
0811 07:11:31.120283 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.120313 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 07:11:31.120450 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
I20260811 07:11:31.134335 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 1
I20260811 07:11:31.134865 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.134718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51918)
0811 07:11:31.134781 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.134829 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 07:11:31.135268 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.135171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51920)
0811 07:11:31.135219 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.135248 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:11:31.135389 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
I20260811 07:11:31.151614 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 1
I20260811 07:11:31.152103 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.151932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51922)
0811 07:11:31.152015 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.152064 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 07:11:31.152510 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.152413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51928)
0811 07:11:31.152458 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.152488 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 07:11:31.152619 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
W20260811 07:11:31.190136 10353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:31.190361 10353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:31.190377 10353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:31.190397 10353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:31.191558 10353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:31.191601 10353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:31.191615 10353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:31.191628 10353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:31.191640 10353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:31.193204 10353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:33075,127.3.236.189:36657,127.3.236.188:38909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:38909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:31.193472 10353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:31.193743 10353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:31.199376 10366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:31.199361 10359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:31.199362 10363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:31.199378 10353 server_base.cc:1034] running on GCE node
I20260811 07:11:31.199810 10353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:31.200122 10353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:31.201261 10353 hybrid_clock.cc:648] HybridClock initialized: now 1786432291201239 us; error 39 us; skew 500 ppm
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:31.203539 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 2
I20260811 07:11:31.203936 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.203818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51932)
0811 07:11:31.203878 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.203910 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 07:11:31.203933 10353 init.cc:377] Logged in from keytab as kudu/127.3.236.188@KRBTEST.COM (short username kudu)
I20260811 07:11:31.204303 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.204190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51946)
0811 07:11:31.204224 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.204274 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260811 07:11:31.204406 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
I20260811 07:11:31.204485 10353 webserver.cc:460] Webserver started at http://127.3.236.188:40771/ using document root <none> and password file <none>
I20260811 07:11:31.204746 10353 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:31.204792 10353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:31.204946 10353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:31.206007 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data/instance:
uuid: "f56a7c04745046a583e005406f2516d0"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "07aefa9bf9ed02c41ba67bc294a407c6"
server_key_iv: "965c6bfa97b2a738e8f08d878ad750e7"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.206377 10353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal/instance:
uuid: "f56a7c04745046a583e005406f2516d0"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "07aefa9bf9ed02c41ba67bc294a407c6"
server_key_iv: "965c6bfa97b2a738e8f08d878ad750e7"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.207713 10353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 07:11:31.208473 10375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:31.208650 10353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 07:11:31.208738 10353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "f56a7c04745046a583e005406f2516d0"
format_stamp: "Formatted at 2026-08-11 07:11:31 on dist-test-slave-2d19"
server_key: "07aefa9bf9ed02c41ba67bc294a407c6"
server_key_iv: "965c6bfa97b2a738e8f08d878ad750e7"
server_key_version: "encryptionkey@0"
I20260811 07:11:31.208792 10353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:31.227747 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 2
I20260811 07:11:31.228164 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.228043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51958)
0811 07:11:31.228098 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.228135 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:11:31.228571 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.228478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51972)
0811 07:11:31.228525 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.228551 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:11:31.228691 10214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:38909: Network error: Client connection negotiation failed: client connection to 127.3.236.188:38909: connect: Connection refused (error 111)
I20260811 07:11:31.243579 10353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:31.246076 10353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:31.246246 10353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:31.250573 10353 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.188:38909
I20260811 07:11:31.250631 10458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.188:38909 every 8 connection(s)
I20260811 07:11:31.250979 10353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
I20260811 07:11:31.252027 10459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:31.253700 10459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.255316 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10353
I20260811 07:11:31.255415 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786432006544353-4018-0/minicluster-data/master-2/wal/instance
I20260811 07:11:31.255652 4018 external_mini_cluster.cc:1428] Setting key 2d84d0b1d3c728ee318c51e8be8e2dec
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:11:31.264736 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.254450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.3.236.190:33075)
0811 07:11:31.254515 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.254517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.254734 (+ 217us) server_negotiation.cc:407] Connection header received
0811 07:11:31.255197 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.255198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.255212 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.255290 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.256066 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.256872 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.258230 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.258482 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.258556 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.260923 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.260925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.260925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.260933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.263138 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.263728 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.263729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.263730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.263784 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.264072 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.264073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.264073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.264178 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.264230 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.264565 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.264643 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:11:31.264951 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.254415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:33075 (local address 127.0.0.1:40762)
0811 07:11:31.254603 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.254645 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.255104 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.255356 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.255359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.255727 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.255978 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.255984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.256977 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.256977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.258140 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.258143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.258605 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.258606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.258734 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.259294 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.259305 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.260834 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.263224 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.263234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.263237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.263649 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.263854 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.263855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.263855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.263989 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.264271 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.264278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.264427 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.264517 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.264692 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260811 07:11:31.265594 10459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.265692 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.256074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51976 (local address 127.3.236.188:38909)
0811 07:11:31.256223 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.256227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.256295 (+ 68us) server_negotiation.cc:407] Connection header received
0811 07:11:31.256325 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.256326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.256342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.256407 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.257131 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.258277 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.259159 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.259616 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.259691 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.262261 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.262270 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.262270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.262284 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.264185 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.264592 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.264596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.264596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.264659 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.264904 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.264905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.264905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.265097 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.265261 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.265458 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.265564 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:31.273000 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.265939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.3.236.189:36657)
0811 07:11:31.265983 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.265984 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.265993 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:11:31.266028 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.266029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.266037 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.266081 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.266409 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.267180 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.268088 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.268417 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.268491 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.270275 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.270277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.270277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.270286 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.271724 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.272081 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.272083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.272083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.272241 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.272472 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.272472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.272473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.272593 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.272666 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.272844 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.272915 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:11:31.272998 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.265881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:36657 (local address 127.0.0.1:52860)
0811 07:11:31.265917 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.265928 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.265971 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.266086 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.266086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.266237 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.266305 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.266307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.267258 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.267259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.268004 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.268006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.268489 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.268490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.268544 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.268950 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.268955 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.270188 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.271813 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.271814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.271814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.271999 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.272335 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.272336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.272337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.272404 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.272671 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.272672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.272717 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.272803 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.272903 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 07:11:31.273450 10459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:31.274317 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.266565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.3.236.190:33075)
0811 07:11:31.266599 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.266600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.266607 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:11:31.266627 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.266628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.266636 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.266687 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.267212 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.267936 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.268914 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.269199 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.269272 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.271625 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.271626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.271627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.271634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.273156 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.273530 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.273532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.273532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.273611 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.273808 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.273809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.273810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.273924 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.274107 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.274143 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.274220 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:11:31.280635 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.273727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51978)
0811 07:11:31.273778 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.273788 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.273834 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.273956 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.273957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.274093 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.274117 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.274118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.274965 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.274966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.275420 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.275421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.275900 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.275900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.275951 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.276388 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.276393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.277803 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.279530 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.279531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.279532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.279707 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.279929 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.279930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.279931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.280004 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.280278 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.280279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.280317 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.280422 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.280523 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 07:11:31.280689 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.273779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.3.236.188:38909)
0811 07:11:31.273811 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.273813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.273820 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:11:31.273890 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.273891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.273901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.273944 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.274190 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.274889 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.275489 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.275808 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.275897 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.277916 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.277917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.277917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.277924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.279422 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.279803 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.279804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.279804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.279852 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.280085 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.280086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.280086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.280182 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.280280 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.280447 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.280573 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":19}
I20260811 07:11:31.282553 10459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: Bootstrap starting.
I20260811 07:11:31.282902 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.274981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.3.236.189:36657)
0811 07:11:31.275019 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.275021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.275576 (+ 555us) server_negotiation.cc:407] Connection header received
0811 07:11:31.275598 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.275599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.275609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.275671 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.276134 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.276838 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.277804 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.278027 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.278105 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.280229 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.280230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.280231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.280237 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.281796 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.282143 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.282145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.282145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.282201 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.282410 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.282411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.282411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.282526 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.282698 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.282734 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.282805 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 07:11:31.283661 10459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:31.284027 10459 log.cc:824] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:31.284410 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:31.284966 10459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: No bootstrap required, opened a new log
I20260811 07:11:31.285473 10172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f978d7e4308e4b4eb814a86e2d17440d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40772
I20260811 07:11:31.287122 10459 raft_consensus.cc:348] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.287282 10459 raft_consensus.cc:729] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56a7c04745046a583e005406f2516d0, State: Initialized, Role: FOLLOWER
I20260811 07:11:31.287375 10459 consensus_queue.cc:260] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.287721 10459 sys_catalog.cc:564] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:31.287693 10468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.287804 10468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:11:31.289680 10483 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:31.289772 10483 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:31.289978 10353 master_runner.cc:186] Error getting master registration for 127.3.236.190:33075: OK, instance_id { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" instance_seqno: 1786432290883372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:11:31.292917 10353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:11:31.295560 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 3
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:11:31.303500 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.295829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51990)
0811 07:11:31.295889 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.295909 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.295956 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.296096 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.296096 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.296304 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.296353 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.296355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.297261 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.297262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.298047 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.298049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.298599 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.298600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.298655 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.299164 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.299169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.300578 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.302361 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.302362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.302363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.302605 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.302812 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.302813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.302813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.302876 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.303182 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.303183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.303228 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.303295 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.303388 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 07:11:31.303504 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.295920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.3.236.188:38909)
0811 07:11:31.295973 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.295975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.295984 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:11:31.296007 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.296007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.296017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.296090 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.296427 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.297164 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.298144 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.298497 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.298591 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.300665 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.300666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.300666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.300674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.302249 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.302679 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.302680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.302680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.302741 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.302941 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.302942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.302942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.303105 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.303185 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.303338 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.303416 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:11:31.305511 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: Bootstrap starting.
I20260811 07:11:31.306282 10342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:31.306594 10342 log.cc:824] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:31.307488 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: No bootstrap required, opened a new log
I20260811 07:11:31.309549 10342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.309698 10342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273e38d86e4b4dd297662b5f93721721, State: Initialized, Role: FOLLOWER
I20260811 07:11:31.309779 10342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.310118 10342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:31.310079 10487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.310182 10487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:11:31.312122 10501 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:31.312179 10501 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:31.312484 10224 master_runner.cc:186] Error getting master registration for 127.3.236.190:33075: OK, instance_id { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" instance_seqno: 1786432290883372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:11:31.315279 10224 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:11:31.327819 10214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } attempt: 3
Aug 11 07:11:31 dist-test-slave-2d19 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:11:31.337381 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.328190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:38909 (local address 127.0.0.1:51992)
0811 07:11:31.328246 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:31.328277 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:31.328320 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:31.328518 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:31.328518 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:31.328714 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:31.328767 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.328769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.329726 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.329727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.330598 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:31.330601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.331079 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.331080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:31.331133 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.331627 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:31.331632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:31.333308 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:31.336135 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.336136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.336137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.336392 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.336697 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:31.336698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:31.336698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:31.336767 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:31.337062 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:31.337063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:31.337115 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 07:11:31.337178 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 07:11:31.337276 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 07:11:31.337433 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.328318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.3.236.188:38909)
0811 07:11:31.328360 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.328362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.328372 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:11:31.328399 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.328400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.328419 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.328499 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.328854 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.329653 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.330697 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.331000 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.331073 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.333414 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.333415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.333416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.333423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.335975 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.336478 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.336479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.336480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.336565 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.336845 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.336846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.336847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.336980 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.337041 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.337247 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.337330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:31.339412 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: Bootstrap starting.
I20260811 07:11:31.340163 10214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:31.340484 10214 log.cc:824] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:31.341224 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: No bootstrap required, opened a new log
I20260811 07:11:31.342592 10214 raft_consensus.cc:348] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.342743 10214 raft_consensus.cc:729] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f978d7e4308e4b4eb814a86e2d17440d, State: Initialized, Role: FOLLOWER
I20260811 07:11:31.342825 10214 consensus_queue.cc:260] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.343168 10214 sys_catalog.cc:564] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:31.343187 10172 raft_consensus.cc:482] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:11:31.343259 10172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:31.344172 10172 raft_consensus.cc:504] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.344440 10172 leader_election.cc:288] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [CANDIDATE]: Term 1 election: Requested vote from peers 273e38d86e4b4dd297662b5f93721721 (127.3.236.189:36657), f56a7c04745046a583e005406f2516d0 (127.3.236.188:38909)
I20260811 07:11:31.345023 10302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f978d7e4308e4b4eb814a86e2d17440d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "273e38d86e4b4dd297662b5f93721721"
I20260811 07:11:31.345170 10302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:31.346112 10302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f978d7e4308e4b4eb814a86e2d17440d in term 1.
I20260811 07:11:31.346629 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f978d7e4308e4b4eb814a86e2d17440d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f56a7c04745046a583e005406f2516d0"
I20260811 07:11:31.346756 10423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:31.347642 10423 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f978d7e4308e4b4eb814a86e2d17440d in term 1.
I20260811 07:11:31.344733 10506 sys_catalog.cc:455] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.346376 10119 leader_election.cc:302] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273e38d86e4b4dd297662b5f93721721, f978d7e4308e4b4eb814a86e2d17440d; no voters:
I20260811 07:11:31.349059 10506 sys_catalog.cc:458] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:31.349298 10506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:31.349442 10506 raft_consensus.cc:686] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [term 1 LEADER]: Becoming Leader. State: Replica: f978d7e4308e4b4eb814a86e2d17440d, State: Running, Role: LEADER
I20260811 07:11:31.349524 10506 consensus_queue.cc:237] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } }
I20260811 07:11:31.349931 10506 sys_catalog.cc:455] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f978d7e4308e4b4eb814a86e2d17440d. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.350009 10506 sys_catalog.cc:458] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: This master's current role is: LEADER
W20260811 07:11:31.350591 10522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:31.350754 10522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:31.353108 10543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:31.353307 10543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:31.359138 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.348809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.3.236.189:36657)
0811 07:11:31.348853 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.348855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.351351 (+ 2496us) server_negotiation.cc:407] Connection header received
0811 07:11:31.351454 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.351455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.351468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.351497 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.352381 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.353145 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.354239 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.354537 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.354579 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.356206 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.356208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.356208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.356215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.357871 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.358282 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.358283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.358283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.358337 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.358530 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.358531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.358531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.358658 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.358746 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.358954 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.359032 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:31.362901 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.349265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.3.236.188:38909)
0811 07:11:31.350908 (+ 1643us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.350910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.352531 (+ 1621us) server_negotiation.cc:407] Connection header received
0811 07:11:31.352630 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.352631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.352642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.352672 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.353605 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.354285 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.356126 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.356442 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.356485 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.359983 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.359985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.359985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.359992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.361643 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.362053 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.362054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.362054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.362111 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.362324 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.362324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.362325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.362453 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.362523 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.362725 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.362804 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1621}
I20260811 07:11:31.365604 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:31.351311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.3.236.190:33075)
0811 07:11:31.351456 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.351458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.355327 (+ 3869us) server_negotiation.cc:407] Connection header received
0811 07:11:31.355439 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.355440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.355450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.355478 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.358093 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.358810 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.360083 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:31.360688 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:31.360733 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:31.362339 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:31.362341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:31.362341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:31.362348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:31.364346 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.364763 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.364764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.364765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.364828 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:31.365048 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:31.365049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:31.365050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:31.365183 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:31.365279 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:31.365433 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 07:11:31.365512 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 07:11:31.369216 10423 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [term 1 FOLLOWER]: Refusing update from remote peer f978d7e4308e4b4eb814a86e2d17440d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:31.369597 10537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:31.371253 10468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f978d7e4308e4b4eb814a86e2d17440d. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.371345 10468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:31.372015 10302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [term 1 FOLLOWER]: Refusing update from remote peer f978d7e4308e4b4eb814a86e2d17440d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:31.372495 10506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [LEADER]: Connected to new peer: Peer: permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:31.373804 10487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f978d7e4308e4b4eb814a86e2d17440d. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.373914 10487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:31.374960 10537 sys_catalog.cc:455] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.375051 10537 sys_catalog.cc:458] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:31.375419 10468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.375502 10468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:31.375633 10550 mvcc.cc:205] Tried to move back new op lower bound from 7317226665445965824 to 7317226665368449024. Current Snapshot: MvccSnapshot[applied={T|T < 7317226665445965824}]
I20260811 07:11:31.375718 10548 mvcc.cc:205] Tried to move back new op lower bound from 7317226665445965824 to 7317226665368449024. Current Snapshot: MvccSnapshot[applied={T|T < 7317226665445965824}]
I20260811 07:11:31.376571 10543 catalog_manager.cc:1347] Generated new cluster ID: ec50be99f11a4c629f590fe09f8f56a9
I20260811 07:11:31.376619 10543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:31.385653 10506 sys_catalog.cc:455] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.385782 10506 sys_catalog.cc:458] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:31.386250 10487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f978d7e4308e4b4eb814a86e2d17440d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f978d7e4308e4b4eb814a86e2d17440d" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 33075 } } peers { permanent_uuid: "273e38d86e4b4dd297662b5f93721721" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 36657 } } peers { permanent_uuid: "f56a7c04745046a583e005406f2516d0" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 38909 } } }
I20260811 07:11:31.386355 10487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:31.386456 10554 mvcc.cc:205] Tried to move back new op lower bound from 7317226665445965824 to 7317226665368449024. Current Snapshot: MvccSnapshot[applied={T|T < 7317226665445965824}]
I20260811 07:11:31.416972 10543 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:31.419072 10543 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:31.442189 10543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f978d7e4308e4b4eb814a86e2d17440d: Generated new TSK 0
I20260811 07:11:31.443315 10543 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:11:31.502403 10219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.499218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.3.236.190:33075)
0811 07:11:31.499301 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.499304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.500113 (+ 809us) server_negotiation.cc:407] Connection header received
0811 07:11:31.500965 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.500967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.500991 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.501057 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.502258 (+ 1201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40796: BlockingRecv error: recv got EOF from 127.0.0.1:40796 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 07:11:31.506052 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.500569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.3.236.189:36657)
0811 07:11:31.500665 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.500667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.504697 (+ 4030us) server_negotiation.cc:407] Connection header received
0811 07:11:31.504963 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.504965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.505002 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.505084 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.505896 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52886: BlockingRecv error: recv got EOF from 127.0.0.1:52886 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 07:11:31.508114 10466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.503704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.3.236.188:38909)
0811 07:11:31.503785 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.503796 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.506860 (+ 3064us) server_negotiation.cc:407] Connection header received
0811 07:11:31.507176 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.507177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.507190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.507222 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.507996 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52014: BlockingRecv error: recv got EOF from 127.0.0.1:52014 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 07:11:31.514608 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.511521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.3.236.189:36657)
0811 07:11:31.511644 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.511646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.513057 (+ 1411us) server_negotiation.cc:407] Connection header received
0811 07:11:31.513320 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.513321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.513356 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.513449 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.514463 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52896: BlockingRecv error: recv got EOF from 127.0.0.1:52896 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 07:11:31.520041 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.518416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.3.236.189:36657)
0811 07:11:31.518497 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.518498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.518958 (+ 460us) server_negotiation.cc:407] Connection header received
0811 07:11:31.519194 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.519195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.519208 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.519276 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.519957 (+ 681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52898: BlockingRecv error: recv got EOF from 127.0.0.1:52898 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 07:11:31.526464 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.524951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.3.236.189:36657)
0811 07:11:31.525012 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.525014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.525407 (+ 393us) server_negotiation.cc:407] Connection header received
0811 07:11:31.525660 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.525661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.525682 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.525774 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.526402 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52900: BlockingRecv error: recv got EOF from 127.0.0.1:52900 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 07:11:31.534327 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.532704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.3.236.189:36657)
0811 07:11:31.532772 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.532783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.533261 (+ 478us) server_negotiation.cc:407] Connection header received
0811 07:11:31.533474 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.533475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.533497 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.533542 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.534262 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52908: BlockingRecv error: recv got EOF from 127.0.0.1:52908 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 07:11:31.546510 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.544892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.3.236.189:36657)
0811 07:11:31.544970 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.544972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.545516 (+ 544us) server_negotiation.cc:407] Connection header received
0811 07:11:31.545711 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.545717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.545740 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.545793 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.546422 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52910: BlockingRecv error: recv got EOF from 127.0.0.1:52910 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 07:11:31.567771 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.565433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.3.236.189:36657)
0811 07:11:31.565526 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.565535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.566450 (+ 915us) server_negotiation.cc:407] Connection header received
0811 07:11:31.566718 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.566720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.566750 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.566825 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.567664 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52924: BlockingRecv error: recv got EOF from 127.0.0.1:52924 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 07:11:31.606714 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.603916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.3.236.189:36657)
0811 07:11:31.604046 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.604048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.605207 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 07:11:31.605400 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.605401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.605423 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.605590 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.606611 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52936: BlockingRecv error: recv got EOF from 127.0.0.1:52936 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 07:11:31.676725 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.674680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.3.236.189:36657)
0811 07:11:31.674817 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.674820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.675104 (+ 284us) server_negotiation.cc:407] Connection header received
0811 07:11:31.675440 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.675446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.675479 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.675677 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.676556 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52938: BlockingRecv error: recv got EOF from 127.0.0.1:52938 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 07:11:31.808199 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:31.806659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.3.236.189:36657)
0811 07:11:31.806787 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:31.806791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:31.807024 (+ 233us) server_negotiation.cc:407] Connection header received
0811 07:11:31.807270 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:31.807271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:31.807300 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:31.807410 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:31.808030 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52954: BlockingRecv error: recv got EOF from 127.0.0.1:52954 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 07:11:32.068679 10349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:32.066940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52960 (local address 127.3.236.189:36657)
0811 07:11:32.067071 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.067074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.067285 (+ 211us) server_negotiation.cc:407] Connection header received
0811 07:11:32.067577 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.067578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.067597 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.067716 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:32.068547 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52960: BlockingRecv error: recv got EOF from 127.0.0.1:52960 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 07:11:32.291013 10483 catalog_manager.cc:1259] Loaded cluster ID: ec50be99f11a4c629f590fe09f8f56a9
I20260811 07:11:32.291164 10483 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: loading cluster ID for follower catalog manager: success
I20260811 07:11:32.294332 10483 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: acquiring CA information for follower catalog manager: success
I20260811 07:11:32.295290 10483 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f56a7c04745046a583e005406f2516d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:11:32.312999 10501 catalog_manager.cc:1259] Loaded cluster ID: ec50be99f11a4c629f590fe09f8f56a9
I20260811 07:11:32.313194 10501 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: loading cluster ID for follower catalog manager: success
I20260811 07:11:32.314822 10501 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: acquiring CA information for follower catalog manager: success
I20260811 07:11:32.315357 10501 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 273e38d86e4b4dd297662b5f93721721: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:11:32.584731 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:32.582139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.3.236.189:36657)
0811 07:11:32.582248 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.582250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.582260 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:11:32.582294 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.582295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.582348 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.582455 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:32.582618 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.583131 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.583855 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.584076 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.584132 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:32.584326 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:32.584484 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:32.584561 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:32.584603 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 07:11:32.584621 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
I20260811 07:11:32.589556 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:32.586930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.3.236.188:38909)
0811 07:11:32.587060 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.587063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.587074 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:11:32.587146 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.587147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.587182 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.587254 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:32.587374 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.587866 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.588673 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.588863 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.588919 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:32.589162 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:32.589268 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:32.589331 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:32.589426 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 07:11:32.589444 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260811 07:11:32.593852 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:32.591461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.3.236.189:36657)
0811 07:11:32.591504 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.591507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.591515 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:11:32.591541 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.591542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.591554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.591633 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:32.591775 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.592241 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.593054 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.593346 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.593406 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:32.593580 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:32.593614 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:32.593666 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:32.593727 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:11:32.593752 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 07:11:32.594187 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:32.591424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.3.236.188:38909)
0811 07:11:32.591462 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.591465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.591471 (+ 6us) server_negotiation.cc:407] Connection header received
0811 07:11:32.591506 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.591506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.591515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.591593 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:32.591687 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.592383 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.593412 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.593707 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.593767 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:32.593942 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:32.593975 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:32.594029 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:32.594093 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:11:32.594115 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 07:11:32.594980 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:32.591477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.3.236.190:33075)
0811 07:11:32.591746 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:32.591748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:32.591756 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:11:32.591796 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:32.591797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:32.591847 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:32.592111 (+ 264us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:32.592150 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.592889 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.593775 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:32.594311 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:32.594428 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:32.594607 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:32.594737 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:32.594807 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:32.594846 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 07:11:32.594866 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":71,"threads_started":1}
I20260811 07:11:32.596855 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10086
I20260811 07:11:32.611963 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10224
I20260811 07:11:32.626623 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10353
2026-08-11T07:11:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3377 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:11:32 dist-test-slave-2d19 krb5kdc[10695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:11:32 dist-test-slave-2d19 krb5kdc[10695](info): set up 2 sockets
Aug 11 07:11:32 dist-test-slave-2d19 krb5kdc[10695](info): commencing operation
krb5kdc: starting...
W20260811 07:11:33.934994 4018 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/test-user.keytab.
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:11:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
Entry for principal kudu/127.3.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab.
I20260811 07:11:34.067271 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46711
--webserver_interface=127.3.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:11:34.155473 10715 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:34.155745 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:34.155779 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:34.155823 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:34.157059 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:34.157131 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:34.157155 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:34.157176 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:34.157205 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:34.158681 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.190.keytab
--principal=kudu/127.3.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.190:46711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:34.158977 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:34.159214 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:34.164506 10725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:34.164489 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:34.164494 10728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:34.164806 10715 server_base.cc:1034] running on GCE node
I20260811 07:11:34.165093 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:34.165397 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:34.166584 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786432294166560 us; error 45 us; skew 500 ppm
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:34.169569 10715 init.cc:377] Logged in from keytab as kudu/127.3.236.190@KRBTEST.COM (short username kudu)
I20260811 07:11:34.170043 10715 webserver.cc:460] Webserver started at http://127.3.236.190:46001/ using document root <none> and password file <none>
I20260811 07:11:34.170259 10715 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:34.170305 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:34.170403 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:34.171581 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data/instance:
uuid: "e8bd376ab9c14939b59a19263a88b6ed"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "42894f278b44dea0ccefaea71bdb67d3"
server_key_iv: "caf1f9549afa1d51b1b2576089f2ae1f"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.171964 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance:
uuid: "e8bd376ab9c14939b59a19263a88b6ed"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "42894f278b44dea0ccefaea71bdb67d3"
server_key_iv: "caf1f9549afa1d51b1b2576089f2ae1f"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.173588 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260811 07:11:34.174578 10740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:34.174785 10715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:11:34.174882 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal
uuid: "e8bd376ab9c14939b59a19263a88b6ed"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "42894f278b44dea0ccefaea71bdb67d3"
server_key_iv: "caf1f9549afa1d51b1b2576089f2ae1f"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.174939 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:34.189083 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:34.191079 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:34.191195 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:34.195382 10715 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.190:46711
I20260811 07:11:34.195459 10831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.190:46711 every 8 connection(s)
I20260811 07:11:34.195818 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/data/info.pb
I20260811 07:11:34.196928 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:34.198663 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:34.203524 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10715
I20260811 07:11:34.203658 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-0/wal/instance
I20260811 07:11:34.203943 4018 external_mini_cluster.cc:1428] Setting key 68a3650da16ef48ae6c5848d31f14df9
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:34.209189 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.199430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.3.236.190:46711)
0811 07:11:34.199636 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.199640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.199749 (+ 109us) server_negotiation.cc:407] Connection header received
0811 07:11:34.200287 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.200290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.200322 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.200420 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.200999 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.201817 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.202712 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.203011 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.203090 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.205548 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.205557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.205557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.205566 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.207346 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.207829 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.207832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.207832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.207917 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.208181 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.208182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.208182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.208474 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.208547 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.208826 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.209025 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":173,"thread_start_us":132,"threads_started":1}
I20260811 07:11:34.209200 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.199401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46711 (local address 127.0.0.1:60046)
0811 07:11:34.199587 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.199630 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.200112 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.200479 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.200482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.200740 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.200897 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.200901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.201902 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.201903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.202635 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.202637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.203126 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.203128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.203188 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.203792 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.203804 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.205456 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.207446 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.207453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.207455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.207741 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.207989 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.207990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.207990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.208110 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.208556 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.208564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.208646 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.208760 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.208962 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260811 07:11:34.210089 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:34.210475 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.210333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36008)
0811 07:11:34.210386 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.210442 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:11:34.210656 10741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
I20260811 07:11:34.211187 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.211077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36022)
0811 07:11:34.211127 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.211160 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 07:11:34.211337 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:38375: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
Entry for principal kudu/127.3.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab.
I20260811 07:11:34.231750 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:38375
--webserver_interface=127.3.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:11:34.260577 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } attempt: 1
I20260811 07:11:34.261090 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.260920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36034)
0811 07:11:34.260997 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.261049 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 07:11:34.261471 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.261359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36042)
0811 07:11:34.261413 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.261447 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:11:34.261575 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:38375: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
W20260811 07:11:34.315692 10847 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:34.315910 10847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:34.315927 10847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:34.315949 10847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:34.317129 10847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:34.317174 10847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:34.317188 10847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:34.317201 10847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:34.317214 10847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:34.318720 10847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.189.keytab
--principal=kudu/127.3.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.189:38375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:34.318980 10847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:34.319219 10847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:34.324374 10857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:34.324394 10855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:34.324474 10847 server_base.cc:1034] running on GCE node
W20260811 07:11:34.324455 10860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:34.324910 10847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:34.325222 10847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:34.326414 10847 hybrid_clock.cc:648] HybridClock initialized: now 1786432294326397 us; error 50 us; skew 500 ppm
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:34.329011 10847 init.cc:377] Logged in from keytab as kudu/127.3.236.189@KRBTEST.COM (short username kudu)
I20260811 07:11:34.329432 10847 webserver.cc:460] Webserver started at http://127.3.236.189:36169/ using document root <none> and password file <none>
I20260811 07:11:34.329663 10847 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:34.329746 10847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:34.329718 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } attempt: 2
I20260811 07:11:34.329874 10847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:34.330116 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.329975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36046)
0811 07:11:34.330046 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.330083 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:11:34.330554 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.330410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36050)
0811 07:11:34.330496 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.330527 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 07:11:34.330687 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.189:38375: Network error: Client connection negotiation failed: client connection to 127.3.236.189:38375: connect: Connection refused (error 111)
I20260811 07:11:34.331223 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data/instance:
uuid: "e195e2c6f84445a1977282ebd5ba4a5d"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "7dd5f5abc029e107202024d90d941f1b"
server_key_iv: "cf1d998d91e984013e1782bda45131b7"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.331589 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal/instance:
uuid: "e195e2c6f84445a1977282ebd5ba4a5d"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "7dd5f5abc029e107202024d90d941f1b"
server_key_iv: "cf1d998d91e984013e1782bda45131b7"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.333022 10847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:34.333801 10868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:34.334121 10847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:34.334236 10847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal
uuid: "e195e2c6f84445a1977282ebd5ba4a5d"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "7dd5f5abc029e107202024d90d941f1b"
server_key_iv: "cf1d998d91e984013e1782bda45131b7"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.334312 10847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:34.367693 10847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:34.370029 10847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:34.370213 10847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:34.374545 10847 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.189:38375
I20260811 07:11:34.374583 10956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.189:38375 every 8 connection(s)
I20260811 07:11:34.374959 10847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/data/info.pb
I20260811 07:11:34.376098 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:34.377221 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10847
I20260811 07:11:34.377354 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-1/wal/instance
I20260811 07:11:34.377578 4018 external_mini_cluster.cc:1428] Setting key 57ffdf81ea03cb2d0a0a0ef327be3531
I20260811 07:11:34.378122 10957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.236.188@KRBTEST.COM; defaulting to no policy
I20260811 07:11:34.389915 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.378969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.3.236.190:46711)
0811 07:11:34.379035 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.379038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.379265 (+ 227us) server_negotiation.cc:407] Connection header received
0811 07:11:34.380084 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.380085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.380100 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.380159 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.381147 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.381787 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.383571 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.383794 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.383856 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.386685 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.386687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.386688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.386697 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.388461 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.388890 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.388891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.388892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.389005 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.389291 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.389291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.389292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.389407 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.389468 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.389774 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.389845 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:11:34.390202 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.378942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46711 (local address 127.0.0.1:60062)
0811 07:11:34.379121 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.379160 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.379952 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.380246 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.380274 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.380708 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.381046 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.381054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.381890 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.381892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.383468 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.383472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.383922 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.383924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.384123 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.384792 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.384806 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.386573 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.388560 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.388568 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.388570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.388804 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.389087 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.389088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.389088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.389217 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.389499 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.389506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.389639 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.389750 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.389884 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260811 07:11:34.390885 10957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.236.188@KRBTEST.COM" created.
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:11:34.399032 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.391197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36056)
0811 07:11:34.391256 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.391267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.391314 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.391639 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.391640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.391793 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.391842 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.391843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.392918 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.392919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.393480 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.393481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.393998 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.393999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.394069 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.394545 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.394550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.395795 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.397903 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.397904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.397905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.398075 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.398317 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.398318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.398318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.398382 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.398693 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.398694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.398733 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.398805 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.398910 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 07:11:34.399101 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.391291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.3.236.189:38375)
0811 07:11:34.391479 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.391483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.391501 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:11:34.391520 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.391522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.391533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.391625 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.391991 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.392811 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.393572 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.393913 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.394001 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.395895 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.395904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.395904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.395914 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.397698 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.398175 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.398179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.398179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.398244 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.398464 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.398465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.398465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.398618 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.398716 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.398837 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.398972 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260811 07:11:34.400013 10957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:34.400477 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.400340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52406)
0811 07:11:34.400396 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.400445 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 07:11:34.400612 10875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
I20260811 07:11:34.401139 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.401038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52408)
0811 07:11:34.401090 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.401115 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:11:34.401252 10957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
Entry for principal kudu/127.3.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab.
I20260811 07:11:34.410985 4018 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhrK1s3/build/release/bin/kudu
/tmp/dist-test-taskhrK1s3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:37679
--webserver_interface=127.3.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.236.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:11:34.421862 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } attempt: 3
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:11:34.429430 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.422245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36058)
0811 07:11:34.422373 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.422393 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.422453 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.422613 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.422614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.422776 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.422843 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.422845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.423804 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.423805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.424415 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.424418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.424934 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.424935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.424999 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.425466 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.425470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.426680 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.428502 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.428503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.428503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.428671 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.428871 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.428871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.428872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.428929 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.429191 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.429192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.429230 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.429285 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.429359 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 07:11:34.429502 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.422347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.3.236.189:38375)
0811 07:11:34.422411 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.422414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.422441 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:11:34.422526 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.422527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.422540 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.422596 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.422914 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.423703 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.424493 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.424852 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.424926 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.426766 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.426767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.426768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.426775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.428424 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.428744 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.428745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.428746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.428806 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.428994 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.428995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.428995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.429129 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.429196 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.429332 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.429409 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:11:34.429944 10832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:34.430295 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.430176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52414)
0811 07:11:34.430235 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.430270 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:11:34.430609 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.430508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52428)
0811 07:11:34.430567 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.430591 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 07:11:34.430686 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
I20260811 07:11:34.450492 10957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 1
I20260811 07:11:34.451028 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.450840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52432)
0811 07:11:34.450917 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.450978 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 07:11:34.451453 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.451329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52440)
0811 07:11:34.451384 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.451423 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 07:11:34.451584 10957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
I20260811 07:11:34.461825 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 1
I20260811 07:11:34.462352 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.462212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52456)
0811 07:11:34.462276 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.462319 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 07:11:34.462781 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.462686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52470)
0811 07:11:34.462733 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.462761 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 07:11:34.462888 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
W20260811 07:11:34.499660 10967 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:11:34.499888 10967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:11:34.499907 10967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:11:34.499926 10967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:11:34.501160 10967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:11:34.501206 10967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:11:34.501221 10967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:11:34.501233 10967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:11:34.501245 10967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:11:34.502805 10967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.236.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.236.190:46711,127.3.236.189:38375,127.3.236.188:37679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/krb5kdc/kudu_127.3.236.188.keytab
--principal=kudu/127.3.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.236.188:37679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 06:13:40 UTC on ccycloud.quasar-axwevi.root.comops.site
build id 4579821
I20260811 07:11:34.503084 10967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:11:34.503338 10967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:11:34.508396 10976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:34.508414 10974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:11:34.508396 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:11:34.508740 10967 server_base.cc:1034] running on GCE node
I20260811 07:11:34.509006 10967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:11:34.509265 10967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:11:34.510416 10967 hybrid_clock.cc:648] HybridClock initialized: now 1786432294510407 us; error 31 us; skew 500 ppm
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:11:34.512991 10967 init.cc:377] Logged in from keytab as kudu/127.3.236.188@KRBTEST.COM (short username kudu)
I20260811 07:11:34.513491 10967 webserver.cc:460] Webserver started at http://127.3.236.188:45435/ using document root <none> and password file <none>
I20260811 07:11:34.513720 10967 fs_manager.cc:359] Metadata directory not provided
I20260811 07:11:34.513789 10967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:11:34.513942 10967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:11:34.515034 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data/instance:
uuid: "963a6c4e93da4fcb8cc41b6c8551bf91"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "9e33458d707ad55bd92cc5f7c03e59eb"
server_key_iv: "9d3f46af08a807c2886aae80c337adf1"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.515404 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal/instance:
uuid: "963a6c4e93da4fcb8cc41b6c8551bf91"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "9e33458d707ad55bd92cc5f7c03e59eb"
server_key_iv: "9d3f46af08a807c2886aae80c337adf1"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.516865 10967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 07:11:34.517680 10994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:11:34.517875 10967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:11:34.518013 10967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data,/tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal
uuid: "963a6c4e93da4fcb8cc41b6c8551bf91"
format_stamp: "Formatted at 2026-08-11 07:11:34 on dist-test-slave-2d19"
server_key: "9e33458d707ad55bd92cc5f7c03e59eb"
server_key_iv: "9d3f46af08a807c2886aae80c337adf1"
server_key_version: "encryptionkey@0"
I20260811 07:11:34.518095 10967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:11:34.519752 10957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 2
I20260811 07:11:34.520164 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.520008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52478)
0811 07:11:34.520079 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.520122 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:11:34.520560 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.520452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52480)
0811 07:11:34.520501 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.520531 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 07:11:34.520673 10957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
I20260811 07:11:34.538069 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 2
I20260811 07:11:34.538218 10967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:11:34.538552 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.538420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52492)
0811 07:11:34.538483 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.538524 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 07:11:34.538995 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.538888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52504)
0811 07:11:34.538943 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.538975 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 07:11:34.539127 10832 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.236.188:37679: Network error: Client connection negotiation failed: client connection to 127.3.236.188:37679: connect: Connection refused (error 111)
I20260811 07:11:34.540724 10967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:11:34.540875 10967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:11:34.545189 10967 rpc_server.cc:307] RPC server started. Bound to: 127.3.236.188:37679
I20260811 07:11:34.545264 11074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.236.188:37679 every 8 connection(s)
I20260811 07:11:34.545614 10967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/data/info.pb
I20260811 07:11:34.546702 11077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:11:34.546890 4018 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhrK1s3/build/release/bin/kudu as pid 10967
I20260811 07:11:34.547048 4018 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhrK1s3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786432006544353-4018-0/minicluster-data/master-2/wal/instance
I20260811 07:11:34.547299 4018 external_mini_cluster.cc:1428] Setting key b4196fa75a50ff71f306efddea1473c1
I20260811 07:11:34.549347 11077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:34.558838 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.550089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.3.236.190:46711)
0811 07:11:34.550160 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.550163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.550354 (+ 191us) server_negotiation.cc:407] Connection header received
0811 07:11:34.550430 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.550431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.550442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.550497 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.551091 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.551831 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.553007 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.553252 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.553331 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.556021 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.556023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.556023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.556031 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.557546 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.557969 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.557970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.557971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.558051 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.558266 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.558267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.558267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.558384 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.558453 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.558655 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.558739 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:11:34.558862 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.547847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.3.236.188:37679)
0811 07:11:34.548057 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.548067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.548102 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:11:34.548616 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.548625 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.548712 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.548875 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.549563 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.550945 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.552073 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.552742 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.552850 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.555353 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.555360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.555361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.555367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.557239 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.557632 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.557635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.557636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.557735 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.557953 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.557954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.557955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.558196 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.558352 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.558474 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.558608 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260811 07:11:34.558871 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.550046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.190:46711 (local address 127.0.0.1:60076)
0811 07:11:34.550233 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.550274 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.550361 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.550535 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.550538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.550860 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.551005 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.551007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.551981 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.551982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.552922 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.552924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.553378 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.553380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.553475 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.553961 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.553970 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.555919 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.557651 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.557659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.557661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.557874 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.558129 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.558130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.558130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.558195 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.558458 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.558466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.558535 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.558626 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.558739 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
I20260811 07:11:34.559525 11077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.190@KRBTEST.COM
I20260811 07:11:34.565912 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.559857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.3.236.189:38375)
0811 07:11:34.559901 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.559903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.559912 (+ 9us) server_negotiation.cc:407] Connection header received
0811 07:11:34.559938 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.559939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.559948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.559997 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.560272 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.560731 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.561391 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.561618 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.561705 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.563340 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.563342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.563342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.563349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.564765 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.565121 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.565123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.565123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.565187 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.565396 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.565397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.565397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.565513 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.565587 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.565739 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.565821 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 07:11:34.565912 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.559797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.189:38375 (local address 127.0.0.1:36060)
0811 07:11:34.559839 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.559850 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.559885 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.559999 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.560000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.560142 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.560171 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.560173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.560829 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.560830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.561306 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.561307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.561706 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.561707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.561767 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.562120 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.562124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.563257 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.564863 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.564864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.564865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.565033 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.565260 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.565261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.565262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.565326 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.565580 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.565581 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.565622 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.565713 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.565819 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 07:11:34.566381 11077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:11:34.567399 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.559896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.3.236.190:46711)
0811 07:11:34.559939 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.559941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.559980 (+ 39us) server_negotiation.cc:407] Connection header received
0811 07:11:34.560052 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.560052 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.560062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.560108 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.560629 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.561178 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.561849 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.562149 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.562250 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.564613 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.564615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.564615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.564623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.566123 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.566507 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.566508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.566509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.566570 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.566787 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.566789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.566789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.566922 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.567119 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.567156 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.567225 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.188@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.236.189@KRBTEST.COM
I20260811 07:11:34.573868 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.566661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52528)
0811 07:11:34.566716 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.566743 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.566781 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.566919 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.566920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.567065 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.567088 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.567090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.567832 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.567833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.568499 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.568508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.569004 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.569005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.569103 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.569671 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.569678 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.570908 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.572643 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.572644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.572645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.572826 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.573049 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.573050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.573050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.573113 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.573419 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.573420 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.573491 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.573567 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.573700 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 07:11:34.573930 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.566772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52528 (local address 127.3.236.188:37679)
0811 07:11:34.566810 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.566812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.566819 (+ 7us) server_negotiation.cc:407] Connection header received
0811 07:11:34.566854 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.566855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.566863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.566916 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.567161 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.567753 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.568593 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.568890 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.568951 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.570997 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.570999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.570999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.571006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.572557 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.572906 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.572907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.572907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.572975 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.573201 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.573202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.573203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.573344 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.573414 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.573611 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.573786 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":23}
I20260811 07:11:34.575768 11077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: Bootstrap starting.
I20260811 07:11:34.576154 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.568039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.3.236.189:38375)
0811 07:11:34.568076 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.568078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.568086 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:11:34.568104 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.568105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.568113 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.568174 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.568665 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.569231 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.570133 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.570340 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.570425 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.573104 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.573105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.573105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.573112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.575031 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.575371 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.575372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.575373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.575430 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.575648 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.575649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.575649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.575762 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.575944 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.575981 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.576058 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:11:34.577066 11077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:34.577425 11077 log.cc:824] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:34.577943 4018 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:11:34.578476 11077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: No bootstrap required, opened a new log
I20260811 07:11:34.579212 10786 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e8bd376ab9c14939b59a19263a88b6ed"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60088
I20260811 07:11:34.580391 11077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.580536 11077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 963a6c4e93da4fcb8cc41b6c8551bf91, State: Initialized, Role: FOLLOWER
I20260811 07:11:34.580629 11077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.581008 11077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:34.581012 11087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.581171 11087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:11:34.582960 11113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:34.583055 11113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:34.583272 10967 master_runner.cc:186] Error getting master registration for 127.3.236.190:46711: OK, instance_id { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" instance_seqno: 1786432294194248 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:11:34.585075 10967 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:11:34.611851 10957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 3
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.189@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:11:34.621138 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.612475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.3.236.188:37679)
0811 07:11:34.612555 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.612558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.612570 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:11:34.612606 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.612607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.612620 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.612707 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.613168 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.614025 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.615070 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.615422 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.615505 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.617948 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.617950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.617950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.617960 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.619687 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.620153 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.620154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.620155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.620215 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.620479 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.620480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.620480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.620685 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.620752 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.620941 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.621033 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:11:34.621184 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.612364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52544)
0811 07:11:34.612427 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.612449 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.612504 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.612737 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.612738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.613000 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.613068 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.613071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.614127 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.614128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.614964 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.614966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.615513 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.615514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.615572 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.616124 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.616130 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.617818 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.619787 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.619788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.619789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.620065 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.620329 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.620330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.620330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.620409 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.620758 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.620760 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.620815 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.620913 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.621026 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 07:11:34.623085 10957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: Bootstrap starting.
I20260811 07:11:34.623808 10957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:34.624114 10957 log.cc:824] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:34.625023 10957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: No bootstrap required, opened a new log
I20260811 07:11:34.626566 10957 raft_consensus.cc:348] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.626716 10957 raft_consensus.cc:729] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e195e2c6f84445a1977282ebd5ba4a5d, State: Initialized, Role: FOLLOWER
I20260811 07:11:34.626837 10957 consensus_queue.cc:260] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.627131 11117 sys_catalog.cc:455] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.627207 10957 sys_catalog.cc:564] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:34.627229 11117 sys_catalog.cc:458] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:11:34.629151 11137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:34.629213 11137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:34.629390 10847 master_runner.cc:186] Error getting master registration for 127.3.236.190:46711: OK, instance_id { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" instance_seqno: 1786432294194248 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:11:34.632375 10847 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:11:34.638262 10832 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } attempt: 3
Aug 11 07:11:34 dist-test-slave-2d19 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786432294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.236.190@KRBTEST.COM for kudu/127.3.236.188@KRBTEST.COM
I20260811 07:11:34.646342 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.638607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.236.188:37679 (local address 127.0.0.1:52550)
0811 07:11:34.638672 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 07:11:34.638685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:11:34.638733 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:11:34.638923 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:11:34.638924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:11:34.639108 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:11:34.639153 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.639155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.640058 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.640059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.640658 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:11:34.640661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.641112 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.641112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:11:34.641169 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.641664 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:11:34.641698 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:11:34.643303 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:11:34.645239 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.645240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.645240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.645422 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.645655 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:11:34.645656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:11:34.645656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:11:34.645716 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:11:34.646004 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:11:34.646005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:11:34.646067 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 07:11:34.646132 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 07:11:34.646236 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 07:11:34.646699 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.638681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.3.236.188:37679)
0811 07:11:34.638806 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.638808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.638816 (+ 8us) server_negotiation.cc:407] Connection header received
0811 07:11:34.638844 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.638845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.638857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.638908 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.639246 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.639987 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.640753 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.641051 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.641114 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.643415 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.643418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.643418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.643426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.645158 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.645517 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.645519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.645519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.645580 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.645783 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.645784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.645784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.645923 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.645997 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.646169 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.646295 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 07:11:34.648425 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: Bootstrap starting.
I20260811 07:11:34.649216 10832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: Neither blocks nor log segments found. Creating new log.
I20260811 07:11:34.649541 10832 log.cc:824] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: Log is configured to *not* fsync() on all Append() calls
I20260811 07:11:34.650322 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: No bootstrap required, opened a new log
I20260811 07:11:34.651368 10832 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.651516 10832 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8bd376ab9c14939b59a19263a88b6ed, State: Initialized, Role: FOLLOWER
I20260811 07:11:34.651609 10832 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.651911 11146 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.652038 10786 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:11:34.651984 10832 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:11:34.652096 10786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:34.652057 11146 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:34.652786 10786 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.653087 10786 leader_election.cc:288] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [CANDIDATE]: Term 1 election: Requested vote from peers e195e2c6f84445a1977282ebd5ba4a5d (127.3.236.189:38375), 963a6c4e93da4fcb8cc41b6c8551bf91 (127.3.236.188:37679)
I20260811 07:11:34.653503 10914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8bd376ab9c14939b59a19263a88b6ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e195e2c6f84445a1977282ebd5ba4a5d"
I20260811 07:11:34.653646 10914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:34.654733 10914 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8bd376ab9c14939b59a19263a88b6ed in term 1.
I20260811 07:11:34.654985 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8bd376ab9c14939b59a19263a88b6ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91"
I20260811 07:11:34.655130 11042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:11:34.655827 11042 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8bd376ab9c14939b59a19263a88b6ed in term 1.
I20260811 07:11:34.656080 10745 leader_election.cc:302] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 963a6c4e93da4fcb8cc41b6c8551bf91, e8bd376ab9c14939b59a19263a88b6ed; no voters:
I20260811 07:11:34.659245 10715 master_runner.cc:263] No leader master found from master e8bd376ab9c14939b59a19263a88b6ed
I20260811 07:11:34.659291 10715 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:11:34.659405 11146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:11:34.659615 11146 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [term 1 LEADER]: Becoming Leader. State: Replica: e8bd376ab9c14939b59a19263a88b6ed, State: Running, Role: LEADER
I20260811 07:11:34.659765 11146 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } }
I20260811 07:11:34.660279 11146 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd376ab9c14939b59a19263a88b6ed. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.660420 11146 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: This master's current role is: LEADER
W20260811 07:11:34.666317 11162 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:11:34.666448 11162 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:11:34.668423 11174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:11:34.668644 11174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:11:34.670367 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.660765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.3.236.189:38375)
0811 07:11:34.660802 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.660804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.662019 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 07:11:34.662122 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.662123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.662135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.662169 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.662815 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.663572 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.664591 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.664904 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.664951 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.667117 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.667119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.667119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.667127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.668936 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.669445 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.669446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.669447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.669510 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.669731 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.669732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.669732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.669867 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.669958 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.670183 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.670267 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:11:34.670899 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.657151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.3.236.190:46711)
0811 07:11:34.657192 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.657194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.657806 (+ 612us) server_negotiation.cc:407] Connection header received
0811 07:11:34.657913 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.657914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.657925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.657967 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.658834 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.659537 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.660582 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.660874 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.660921 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.664288 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.664289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.664290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.664298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.666304 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.670013 (+ 3709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.670014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.670014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.670073 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.670282 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.670283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.670284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.670414 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.670490 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.670714 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.670796 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 07:11:34.673246 11042 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [term 1 FOLLOWER]: Refusing update from remote peer e8bd376ab9c14939b59a19263a88b6ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:34.673604 11171 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:34.674515 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:34.657510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.3.236.188:37679)
0811 07:11:34.657546 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.657548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.661571 (+ 4023us) server_negotiation.cc:407] Connection header received
0811 07:11:34.661679 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.661680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.661692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.661723 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.663191 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.663895 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.667807 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:34.668135 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:34.668194 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:34.671424 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:11:34.671425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:11:34.671426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:11:34.671434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:11:34.673086 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.673569 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.673570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.673570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.673633 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:11:34.673952 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:11:34.673953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:11:34.673954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:11:34.674080 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:11:34.674152 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:34.674343 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 07:11:34.674421 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 07:11:34.674998 11087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd376ab9c14939b59a19263a88b6ed. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.675156 11087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:34.675731 10914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [term 1 FOLLOWER]: Refusing update from remote peer e8bd376ab9c14939b59a19263a88b6ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:11:34.676038 11171 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:11:34.677392 11117 sys_catalog.cc:455] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bd376ab9c14939b59a19263a88b6ed. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.677505 11117 sys_catalog.cc:458] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:34.678503 11146 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.678584 11146 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:34.678684 11087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.678764 11087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:34.678925 11180 mvcc.cc:205] Tried to move back new op lower bound from 7317226678978646016 to 7317226678927380480. Current Snapshot: MvccSnapshot[applied={T|T < 7317226678978646016}]
I20260811 07:11:34.679715 11171 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.679808 11171 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed [sys.catalog]: This master's current role is: LEADER
I20260811 07:11:34.680254 11117 sys_catalog.cc:455] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e8bd376ab9c14939b59a19263a88b6ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8bd376ab9c14939b59a19263a88b6ed" member_type: VOTER last_known_addr { host: "127.3.236.190" port: 46711 } } peers { permanent_uuid: "e195e2c6f84445a1977282ebd5ba4a5d" member_type: VOTER last_known_addr { host: "127.3.236.189" port: 38375 } } peers { permanent_uuid: "963a6c4e93da4fcb8cc41b6c8551bf91" member_type: VOTER last_known_addr { host: "127.3.236.188" port: 37679 } } }
I20260811 07:11:34.680365 11117 sys_catalog.cc:458] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:11:34.681036 11174 catalog_manager.cc:1347] Generated new cluster ID: 7cbe0368806149a98f5d8a07833ff014
I20260811 07:11:34.681105 11174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:11:34.684669 11181 mvcc.cc:205] Tried to move back new op lower bound from 7317226678978646016 to 7317226678927380480. Current Snapshot: MvccSnapshot[applied={T|T < 7317226678978646016}]
I20260811 07:11:34.723583 11174 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:11:34.724424 11174 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:11:34.731693 11174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8bd376ab9c14939b59a19263a88b6ed: Generated new TSK 0
I20260811 07:11:34.731933 11174 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:11:34.765291 10838 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.763567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.3.236.190:46711)
0811 07:11:34.763607 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.763609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.764433 (+ 824us) server_negotiation.cc:407] Connection header received
0811 07:11:34.764641 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.764642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.764653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.764685 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.765218 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60096: BlockingRecv error: recv got EOF from 127.0.0.1:60096 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 07:11:34.765928 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.763335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.3.236.189:38375)
0811 07:11:34.763383 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.763385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.764527 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 07:11:34.765459 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.765461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.765473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.765505 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.765852 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36090: BlockingRecv error: recv got EOF from 127.0.0.1:36090 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 07:11:34.766680 11082 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.763896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.3.236.188:37679)
0811 07:11:34.763936 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.763937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.766156 (+ 2219us) server_negotiation.cc:407] Connection header received
0811 07:11:34.766298 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.766299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.766310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.766339 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.766622 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52574: BlockingRecv error: recv got EOF from 127.0.0.1:52574 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 07:11:34.769176 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.768330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.3.236.189:38375)
0811 07:11:34.768368 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.768371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.768648 (+ 277us) server_negotiation.cc:407] Connection header received
0811 07:11:34.768749 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.768750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.768767 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.768796 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.769114 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36092: BlockingRecv error: recv got EOF from 127.0.0.1:36092 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 07:11:34.772174 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.771431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.3.236.189:38375)
0811 07:11:34.771466 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.771467 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.771676 (+ 209us) server_negotiation.cc:407] Connection header received
0811 07:11:34.771768 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.771769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.771779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.771807 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.772131 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36102: BlockingRecv error: recv got EOF from 127.0.0.1:36102 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 07:11:34.776322 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.775395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.3.236.189:38375)
0811 07:11:34.775433 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.775435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.775741 (+ 306us) server_negotiation.cc:407] Connection header received
0811 07:11:34.775846 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.775847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.775858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.775904 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.776244 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36116: BlockingRecv error: recv got EOF from 127.0.0.1:36116 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 07:11:34.782389 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.781532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.3.236.189:38375)
0811 07:11:34.781568 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.781571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.781851 (+ 280us) server_negotiation.cc:407] Connection header received
0811 07:11:34.781964 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.781966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.781978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.782007 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.782342 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36120: BlockingRecv error: recv got EOF from 127.0.0.1:36120 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 07:11:34.792716 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.791863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.3.236.189:38375)
0811 07:11:34.791939 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.791941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.792042 (+ 101us) server_negotiation.cc:407] Connection header received
0811 07:11:34.792125 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.792127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.792140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.792214 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.792656 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36128: BlockingRecv error: recv got EOF from 127.0.0.1:36128 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:11:34.810829 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.810025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.3.236.189:38375)
0811 07:11:34.810076 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.810078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.810207 (+ 129us) server_negotiation.cc:407] Connection header received
0811 07:11:34.810286 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.810287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.810300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.810356 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.810768 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36144: BlockingRecv error: recv got EOF from 127.0.0.1:36144 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 07:11:34.844928 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.844147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.3.236.189:38375)
0811 07:11:34.844230 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.844233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.844352 (+ 119us) server_negotiation.cc:407] Connection header received
0811 07:11:34.844393 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.844394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.844408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.844489 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.844855 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36146: BlockingRecv error: recv got EOF from 127.0.0.1:36146 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 07:11:34.910985 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:34.910316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.3.236.189:38375)
0811 07:11:34.910390 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:34.910392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:34.910448 (+ 56us) server_negotiation.cc:407] Connection header received
0811 07:11:34.910510 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:34.910511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:34.910524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:34.910577 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:34.910903 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36152: BlockingRecv error: recv got EOF from 127.0.0.1:36152 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 07:11:35.041198 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:35.040274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.3.236.189:38375)
0811 07:11:35.040418 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.040421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.040546 (+ 125us) server_negotiation.cc:407] Connection header received
0811 07:11:35.040601 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.040602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.040614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.040670 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:35.041110 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36164: BlockingRecv error: recv got EOF from 127.0.0.1:36164 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260811 07:11:35.299305 10961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:11:35.298458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.3.236.189:38375)
0811 07:11:35.298538 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.298540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.298659 (+ 119us) server_negotiation.cc:407] Connection header received
0811 07:11:35.298696 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.298697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.298709 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.298772 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:11:35.299217 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36170: BlockingRecv error: recv got EOF from 127.0.0.1:36170 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:11:35.583564 11113 catalog_manager.cc:1259] Loaded cluster ID: 7cbe0368806149a98f5d8a07833ff014
I20260811 07:11:35.583654 11113 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: loading cluster ID for follower catalog manager: success
I20260811 07:11:35.584610 11113 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: acquiring CA information for follower catalog manager: success
I20260811 07:11:35.584957 11113 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 963a6c4e93da4fcb8cc41b6c8551bf91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:11:35.629689 11137 catalog_manager.cc:1259] Loaded cluster ID: 7cbe0368806149a98f5d8a07833ff014
I20260811 07:11:35.629770 11137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: loading cluster ID for follower catalog manager: success
I20260811 07:11:35.630685 11137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: acquiring CA information for follower catalog manager: success
I20260811 07:11:35.631026 11137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e195e2c6f84445a1977282ebd5ba4a5d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:11:35.815261 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:35.812580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.3.236.189:38375)
0811 07:11:35.812747 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.812749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.812764 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:11:35.812844 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.812845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.812901 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.813015 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:35.813177 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.813712 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.814484 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.814676 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.814729 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:35.814903 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:35.815019 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:35.815080 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:35.815173 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:11:35.815192 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260811 07:11:35.819446 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:35.817175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.3.236.188:37679)
0811 07:11:35.817276 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.817277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.817301 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:11:35.817338 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.817338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.817383 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.817445 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:35.817559 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.818014 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.818801 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.818984 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.819034 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:35.819180 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:35.819269 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:35.819321 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:35.819370 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:11:35.819387 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":57,"threads_started":1}
I20260811 07:11:35.824302 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:35.821332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.3.236.188:37679)
0811 07:11:35.821380 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.821383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.821706 (+ 323us) server_negotiation.cc:407] Connection header received
0811 07:11:35.821736 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.821737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.821747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.821838 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:35.822119 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.822687 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.823583 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.823848 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.823921 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:35.824030 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:35.824061 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:35.824118 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:35.824155 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 07:11:35.824174 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:11:35.825150 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:35.821270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.3.236.189:38375)
0811 07:11:35.821371 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.821374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.821561 (+ 187us) server_negotiation.cc:407] Connection header received
0811 07:11:35.821731 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.821732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.821742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.821811 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:35.821973 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.822700 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.824326 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.824664 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.824737 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:35.824897 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:35.824931 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:35.824985 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:35.825049 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:11:35.825069 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:11:35.825482 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:11:35.821269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.3.236.190:46711)
0811 07:11:35.821380 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:11:35.821383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:11:35.821613 (+ 230us) server_negotiation.cc:407] Connection header received
0811 07:11:35.821708 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:11:35.821709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:11:35.821761 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:11:35.821835 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:11:35.822050 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.822800 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.823839 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:11:35.824190 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:11:35.824278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:11:35.824988 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:11:35.825084 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:11:35.825188 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:11:35.825351 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 07:11:35.825378 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260811 07:11:35.827193 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10715
I20260811 07:11:35.843796 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10847
I20260811 07:11:35.861438 4018 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhrK1s3/build/release/bin/kudu with pid 10967
2026-08-11T07:11:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3232 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6610 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289324 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST